Skip to content

docs: act via code blog [wip] #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Contributing to Graph Sitter
# Contributing to Codegen

Thank you for your interest in contributing to Graph Sitter! This document outlines the process and guidelines for contributing.
Thank you for your interest in contributing to Codegen! This document outlines the process and guidelines for contributing.

## Contributor License Agreement

By contributing to Graph Sitter, you agree that:
By contributing to Codegen, you agree that:

1. Your contributions will be licensed under the project's license.
2. You have the right to license your contribution under the project's license.
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Codegen

[![Documentation](https://img.shields.io/badge/docs-docs.codegen.com-blue)](https://docs.codegen.com)
[![Slack Community](https://img.shields.io/badge/slack-community-4A154B?logo=slack)](https://community.codegen.com)
[![Twitter Follow](https://img.shields.io/twitter/follow/codegen)](https://twitter.com/codegen)

[Codegen](https://docs.codegen.com) is a python library for manipulating codebases.

Write code that transforms code. Codegen combines the parsing power of [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) with the graph algorithms of [rustworkx](https://github.com/Qiskit/rustworkx) to enable scriptable, multi-language code manipulation at scale.

```python
from codegen import Codebase
Expand All @@ -21,6 +22,8 @@ for function in codebase.functions:
function.move_to_file('deprecated.py')
```

Write code that transforms code. Codegen combines the parsing power of [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) with the graph algorithms of [rustworkx](https://github.com/Qiskit/rustworkx) to enable scriptable, multi-language code manipulation at scale.

## Installation
**This library requires Python 3.12 – 3.13.**
```
Expand All @@ -31,6 +34,7 @@ uv pip install codegen

- [Docs](https://docs.codegen.com)
- [Get Started](https://docs.codegen.com/introduction/getting-started)
- [Contributing](CONTRIBUTING.md)


## Why Codegen?
Expand Down
100 changes: 100 additions & 0 deletions docs/blog/act-via-code.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
title: "Act via Code"
icon: "code"
iconType: "solid"
description: "The path to advanced code manipulation agents"
---

<Frame caption="Voyager (Jim Fan)">
<img src="/images/nether-portal.png" />
</Frame>


# Act via Code

Two and a half years since the launch of the GPT-3 API, code assistants have emerged as the most powerful and practically useful applications of LLMs. The rapid adoption of AI-powered IDEs and prototype builders isn't surprising — code is structured, deterministic, and rich with patterns, making it an ideal domain for machine learning. As model capabilities continue to scale, we're seeing compounding improvements in code understanding and generation.

Yet there's a striking gap between what AI agents can understand and what they can actually do. While they can reason about complex architectural changes, debug intricate issues, and propose sophisticated refactors, they often can't execute these ideas. The ceiling isn't intelligence or context—it's the ability to manipulate code at scale. Large-scale modifications remain unreliable or impossible, not because agents don't understand what to do, but because they lack the right interfaces to do it.

The bottleneck isn't intelligence — it's tooling. By giving AI models the ability to write and execute code that modifies code, we're about to unlock an entire class of tasks that agents already understand but can't yet perform. Code execution environments represent the most expressive tool we could offer an agent—enabling composition, abstraction, and systematic manipulation of complex systems. When paired with ever-improving language models, this will unlock another step function improvement in AI capabilities.

## Beating Minecraft with Code Execution

In mid-2023, a research project called [Voyager](https://voyager.minedojo.org) made waves: it effectively solved Minecraft, performing several multiples better than the prior SOTA on many important dimensions. This was a massive breakthrough — previous reinforcement learning systems had struggled for years with even basic Minecraft tasks.

While the AI community was focused on scaling intelligence, Voyager demonstrated something more fundamental: the right tools can unlock entirely new tiers of capability. The same GPT-4 model that struggled with Minecraft using traditional frameworks achieved remarkable results when allowed to write and execute code. This wasn't about raw intelligence—it was about giving the agent a more expressive way to act.

<Frame>
<img src="/images/voyager-performance.png" />
</Frame>

The breakthrough came from a simple yet powerful insight: let the AI write code. Instead of limiting the agent to primitive "tools," Voyager allowed GPT-4 to write and execute [JS programs](https://github.com/MineDojo/Voyager/tree/main/skill_library/trial2/skill/code) that controlled Minecraft actions through a clean API:

```javascript
async function chopSpruceLogs(bot) {
const spruceLogCount = bot.inventory.count(mcData.itemsByName.spruce_log.id);
const logsToMine = 3 - spruceLogCount;
if (logsToMine > 0) {
bot.chat("Chopping down spruce logs...");
await mineBlock(bot, "spruce_log", logsToMine);
bot.chat("Chopped down 3 spruce logs.");
} else {
bot.chat("Already have 3 spruce logs in inventory.");
}
}
```

This approach transformed the agent's capabilities. Rather than being constrained to atomic actions like `equipItem(...)`, it could create higher-level operations like [`craftShieldWithFurnace()`](https://github.com/MineDojo/Voyager/blob/main/skill_library/trial2/skill/code/craftShieldWithFurnace.js) through composing JS APIs. The system also implemented a memory mechanism, storing successful programs for reuse in similar situations—effectively building its own library of proven solutions it could later refer to and adapt to similar circumstances.

<Frame>
<img src="/images/voyager-retrieval.png" />
</Frame>

As the Voyager authors noted:

<Tip>*"We opt to use code as the action space instead of low-level motor commands because programs can naturally represent temporally extended and compositional actions, which are essential for many long-horizon tasks in Minecraft."*</Tip>

## Code is an Ideal Action Space

The implications of code as an action space extend far beyond gaming. Code provides a uniquely powerful interface between AI and real-world systems. When an agent writes code, it gains several critical advantages over traditional atomic tools.

### Code is Composable
Code is the ultimate composable medium. Agents can build their own tools by combining simpler operations, wrapping any function as a building block for more complex behaviors. This aligns well with what is perhaps LLMs' premier capability: understanding and interpolating between examples to create new solutions.

### Code Constrains the Action Space
APIs can enforce guardrails that keep agents on track. By designing interfaces that make invalid operations impossible to express, we can prevent entire classes of errors before they happen. The type system becomes a powerful tool for shaping agent behavior.

### Code Provides Objective Feedback
Code execution gives immediate, unambiguous feedback. When something goes wrong, you get stack traces and error messages—not just a confidence score. This concrete error signal is invaluable for agents learning to navigate complex systems.

### Code is a Natural Medium for Collaboration
Programs are a shared language between humans and agents. Code explicitly encodes reasoning in a reviewable format, making agent actions transparent and debuggable. There's no magic—just deterministic execution that can be understood, modified, and improved by both humans and AI.

## For Software Engineering

This brings us to software engineering, where we see a massive gap between AI's theoretical capabilities and practical achievements. Many code modification tasks are fundamentally programmatic—dependency analysis, refactors, control flow analysis—yet we lack the tools to express them properly.

Consider how a developer thinks about refactoring: it's rarely about direct text manipulation. Instead, we think in terms of high-level operations: "move this function," "rename this variable everywhere," "split this module." These operations can be encoded into a powerful Python API:

```python
# simple access to high-level code constructs
for component in codebase.jsx_components:
# access detailed code structure and relations
if len(component.usages) == 0:
# powerful edit APIs that handle edge cases
component.rename(component.name + 'Page')
```

This isn't just another code manipulation library—it's a scriptable language server that builds on proven foundations like LSP and codemods, but designed specifically for programmatic analysis and refactoring.

## What does this look like?

At Codegen, we've built exactly this system. Our approach centers on four key principles:

The foundation must be Python, enabling easy composition with existing tools and workflows. Operations must be in-memory for performance, handling large-scale changes efficiently. The system must be open source, allowing developers and AI researchers to extend and enhance it. And perhaps most importantly, it must be thoroughly documented—not just for humans, but for the next generation of AI agents that will build upon it.

## What does this enable?

We've already used this approach to merge hundreds of thousands of lines of code in enterprise codebases. Our tools have automated complex tasks like feature flag deletion, test suite reorganization, import cycle elimination, and dead code removal. But more importantly, we've proven that code-as-action-space isn't just theoretical—it's a practical approach to scaling software engineering.

This is just the beginning. With Codegen, we're providing the foundation for the next generation of code manipulation tools—built for both human developers and AI agents. We believe this approach will fundamentally change how we think about and implement large-scale code changes, making previously impossible tasks not just possible, but routine.
223 changes: 223 additions & 0 deletions docs/blog/codemod-frameworks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
---
title: "Comparing Codemod Frameworks"
sidebarTitle: "Codemod Frameworks"
icon: "code-compare"
iconType: "solid"
---

Code transformation tools have evolved significantly over the years, each offering unique approaches to programmatic code manipulation. Let's explore the strengths and limitations of major frameworks in this space.

## Python's AST Module

Python's built-in Abstract Syntax Tree (AST) module provides the foundation for Python code manipulation.

### Strengths

- Native Python implementation
- No external dependencies
- Full access to Python's syntax tree
- Great for Python-specific transformations

### Limitations

- Python-only
- Low-level API requiring deep AST knowledge
- Manual handling of formatting and comments
- No cross-file awareness

```python
import ast

class NameTransformer(ast.NodeTransformer):
def visit_Name(self, node):
if node.id == 'old_name':
return ast.Name(id='new_name', ctx=node.ctx)
return node
```

## LibCST

Meta's Concrete Syntax Tree library offers a more modern approach to Python code modification.

### Strengths

- Preserves formatting and comments
- Type annotations support
- Visitor pattern API
- Excellent documentation
- Supports codemods at scale

### Limitations

- Python-only
- Steeper learning curve
- Slower than raw AST manipulation
- Memory-intensive for large codebases

```python
import libcst as cst

class NameTransformer(cst.CSTTransformer):
def leave_Name(self, original_node, updated_node):
if original_node.value == "old_name":
return updated_node.with_changes(value="new_name")
return updated_node
```

## jscodeshift

The pioneer of JavaScript codemods, jscodeshift remains a staple in the JS ecosystem.

### Strengths

- Robust JavaScript/TypeScript support
- Rich ecosystem of transforms
- Familiar jQuery-like API
- Battle-tested at scale

### Limitations

- JavaScript/TypeScript only
- Limited type information
- Can be verbose for simple transforms
- Documentation could be better

```javascript
export default function transformer(file, api) {
const j = api.jscodeshift;
return j(file.source)
.find(j.Identifier)
.filter((path) => path.node.name === "old_name")
.replaceWith((path) => j.identifier("new_name"))
.toSource();
}
```

## ts-morph

A TypeScript-first transformation tool with rich type system integration.

### Strengths

- First-class TypeScript support
- Excellent type information access
- High-level, intuitive API
- Great documentation
- Project-wide analysis capabilities

### Limitations

- TypeScript/JavaScript only
- Higher memory usage
- Can be slower for large projects
- More complex setup than alternatives

```typescript
import { Project } from "ts-morph";

const project = new Project();
project.addSourceFileAtPath("src/**/*.ts");
project.getSourceFiles().forEach((sourceFile) => {
sourceFile
.getDescendantsOfKind(SyntaxKind.Identifier)
.filter((node) => node.getText() === "old_name")
.forEach((node) => node.replaceWithText("new_name"));
});
```

## ast-grep

A modern, language-agnostic code searching and rewriting tool.

### Strengths

- Multi-language support
- Fast pattern matching
- Simple YAML-based rules
- Great for quick transformations
- Excellent performance

### Limitations

- Limited complex transformation support
- Newer, less battle-tested
- Smaller ecosystem
- Less granular control

```yaml
rules:
- pattern: old_name
replace: new_name
```

## tree-sitter

The foundation many modern tools build upon, offering lightning-fast parsing and analysis.

### Strengths

- Incredible performance
- Multi-language support
- Incremental parsing
- Language-agnostic design
- Growing ecosystem

### Limitations

- Lower-level API
- Requires language-specific grammars
- Manual handling of transformations
- Steeper learning curve

```javascript
const Parser = require("tree-sitter");
const JavaScript = require("tree-sitter-javascript");

const parser = new Parser();
parser.setLanguage(JavaScript);
const tree = parser.parse('console.log("Hello")');
```

## Choosing the Right Tool

The choice of codemod framework depends heavily on your specific needs:

- **Single Language Focus**: If you're working exclusively with one language, use its specialized tools:

- Python → LibCST
- TypeScript → ts-morph
- JavaScript → jscodeshift

- **Multi-Language Projects**: Consider:

- ast-grep for simple transformations
- tree-sitter for building custom tools
- A combination of specialized tools

- **Scale Considerations**:
- Small projects → Any tool works
- Medium scale → Language-specific tools
- Large scale → Need proper tooling support (LibCST, ts-morph)

## The Future of Codemods

As codebases grow and AI becomes more prevalent, we're seeing a shift toward:

1. **More Intelligent Tools**

- Better type awareness
- Improved cross-file analysis
- AI-assisted transformations

2. **Universal Approaches**

- Language-agnostic frameworks
- Unified transformation APIs
- Better interoperability

3. **Enhanced Developer Experience**
- Simpler APIs
- Better debugging tools
- Richer ecosystems

The ideal codemod framework of the future will likely combine the best aspects of current tools: the type awareness of ts-morph, the simplicity of ast-grep, the performance of tree-sitter, and the reliability of LibCST.
Loading
Loading