Skip to content

feat(cg-10482): codegen notebook #78

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/cli/about.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Codegen CLI"
sidebarTitle: "Overview"
icon: "terminal"
icon: "square-info"
iconType: "solid"
---

Expand Down
74 changes: 74 additions & 0 deletions docs/cli/notebook.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "notebook"
sidebarTitle: "notebook"
description: "Open a Jupyter notebook with the current codebase loaded"
icon: "book"
iconType: "solid"
---

The `notebook` command launches a Jupyter Lab instance with a pre-configured notebook for exploring your codebase.

## Usage

```bash
codegen notebook [--background]
```

### Options

- `--background`: Run Jupyter Lab in the background (default: false)

By default, Jupyter Lab runs in the foreground, making it easy to stop with Ctrl+C. Use `--background` if you want to run it in the background.

## What it Does

This will:

1. Create a Python virtual environment in `.codegen/.venv` if it doesn't exist
2. Install required packages (codegen and jupyterlab)
3. Create a starter notebook in `.codegen/jupyter/tmp.ipynb`
4. Launch Jupyter Lab with the notebook open

The notebook comes pre-configured with:

```python
from codegen import Codebase

# Initialize codebase
codebase = Codebase('../../')
```

<Note>
The notebook is created in `.codegen/jupyter/`, so the relative path `../../`
points to your repository root.
</Note>

## Directory Structure

After running `codegen notebook`, your repository will have this structure:

```
.codegen/
├── .venv/ # Virtual environment for this project
├── jupyter/ # Jupyter notebooks
│ └── tmp.ipynb # Pre-configured notebook
└── config.toml # Project configuration
```

## Examples

```bash
# Run Jupyter in the foreground (default)
codegen notebook

# Run Jupyter in the background
codegen notebook --background

# Initialize a new repository and launch notebook
codegen init && codegen notebook
```

<Tip>
The notebook command will automatically initialize codegen if needed, so you
can run it directly in a new repository.
</Tip>
51 changes: 29 additions & 22 deletions docs/introduction/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,48 +5,55 @@ icon: "bolt"
iconType: "solid"
---

Follow our step-by-step tutorial to get up and running with Codegen in a jupyter notebook.
Follow our step-by-step tutorial to get up and running with Codegen.

## Installation

We recommend using [`uv`](https://github.com/astral-sh/uv) with Python 3.13 for the best experience.
We recommend using [`pipx`](https://pypa.github.io/pipx/) to install Codegen globally. `pipx` is a tool to help you install and run Python applications in isolated environments. This isolation ensures that Codegen's dependencies won't conflict with your other Python projects.

First, install `uv` if you haven't already:
First, install `pipx` if you haven't already:

```bash
brew install uv
brew install pipx
pipx ensurepath # Ensure pipx binaries are on your PATH
```

`cd` into the directory you want to parse:
Then install Codegen globally:

```bash
cd path/to/git/repository
pipx install codegen
```

Create and activate a Python 3.13 virtual environment:
<Note>
This makes the `codegen` command available globally in your terminal, while
keeping its dependencies isolated.
</Note>

```bash
uv venv --python 3.13.0 && source .venv/bin/activate
```
## Quick Start with Jupyter

Install [Codegen from PyPI](https://pypi.org/project/codegen/) using `uv`:
The fastest way to explore a codebase is using Jupyter. Codegen provides a built-in command to set this up:

```bash
uv pip install codegen
```
# Navigate to your repository
cd path/to/git/repository

Install and run Jupyter (optional):
# Initialize codegen and launch Jupyter
codegen init
codegen notebook
```

<Tip>Jupyter notebooks are great for exploration and debugging.</Tip>
This will:

```bash
uv pip install jupyterlab && jupyter lab
```
1. Create a `.codegen/` directory with:
- `.venv/` - A dedicated virtual environment for this project
- `jupyter/` - Jupyter notebooks for exploring your code
- `config.toml` - Project configuration
2. Launch Jupyter Lab with a pre-configured notebook

<Info>
You can also use the CLI if you prefer to work in a terminal. [Learn more
here](/cli/about).
</Info>
<Tip>
The notebook comes pre-configured to load your codebase, so you can start
exploring right away!
</Tip>

## Initializing a Codebase

Expand Down
9 changes: 8 additions & 1 deletion docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,14 @@
},
{
"group": "CLI",
"pages": ["cli/about", "cli/init", "cli/create", "cli/run", "cli/expert"]
"pages": [
"cli/about",
"cli/init",
"cli/notebook",
"cli/create",
"cli/run",
"cli/expert"
]
},
{
"group": "Changelog",
Expand Down
2 changes: 2 additions & 0 deletions src/codegen/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from codegen.cli.commands.list.main import list_command
from codegen.cli.commands.login.main import login_command
from codegen.cli.commands.logout.main import logout_command
from codegen.cli.commands.notebook.main import notebook_command
from codegen.cli.commands.profile.main import profile_command
from codegen.cli.commands.run.main import run_command
from codegen.cli.commands.run_on_pr.main import run_on_pr_command
Expand Down Expand Up @@ -35,6 +36,7 @@ def main():
main.add_command(deploy_command)
main.add_command(style_debug_command)
main.add_command(run_on_pr_command)
main.add_command(notebook_command)


if __name__ == "__main__":
Expand Down
43 changes: 27 additions & 16 deletions src/codegen/cli/commands/init/main.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import subprocess
import sys
from pathlib import Path

import rich
import rich_click as click
import toml

from codegen.cli.auth.decorators import requires_auth
from codegen.cli.auth.constants import CODEGEN_DIR
from codegen.cli.auth.session import CodegenSession
from codegen.cli.commands.init.render import get_success_message
from codegen.cli.git.url import get_git_organization_and_repo
Expand All @@ -15,8 +17,8 @@
@click.command(name="init")
@click.option("--repo-name", type=str, help="The name of the repository")
@click.option("--organization-name", type=str, help="The name of the organization")
@requires_auth
def init_command(session: CodegenSession, repo_name: str | None = None, organization_name: str | None = None):
@click.option("--fetch-docs", is_flag=True, help="Fetch docs and examples (requires auth)")
def init_command(repo_name: str | None = None, organization_name: str | None = None, fetch_docs: bool = False):
"""Initialize or update the Codegen folder."""
# Print a message if not in a git repo
try:
Expand All @@ -30,27 +32,36 @@ def init_command(session: CodegenSession, repo_name: str | None = None, organiza
rich.print(format_command("codegen init"))
sys.exit(1)

codegen_dir = session.codegen_dir
# Only create session if we need to fetch docs
session = CodegenSession() if fetch_docs else None
codegen_dir = Path.cwd() / CODEGEN_DIR if not session else session.codegen_dir
is_update = codegen_dir.exists()

if organization_name is not None:
session.config.organization_name = organization_name
if repo_name is not None:
session.config.repo_name = repo_name
if not session.config.organization_name or not session.config.repo_name:
cwd_org, cwd_repo = get_git_organization_and_repo(session.git_repo)
session.config.organization_name = session.config.organization_name or cwd_org
session.config.repo_name = session.config.repo_name or cwd_repo
session.write_config()
# Only update config if we have a session
if session:
if organization_name is not None:
session.config.organization_name = organization_name
if repo_name is not None:
session.config.repo_name = repo_name
if not session.config.organization_name or not session.config.repo_name:
cwd_org, cwd_repo = get_git_organization_and_repo(session.git_repo)
session.config.organization_name = session.config.organization_name or cwd_org
session.config.repo_name = session.config.repo_name or cwd_repo
session.write_config()

action = "Updating" if is_update else "Initializing"
rich.print("") # Add a newline before the spinner
codegen_dir, docs_dir, examples_dir = initialize_codegen(action=action)
codegen_dir, docs_dir, examples_dir = initialize_codegen(action, session=session, fetch_docs=fetch_docs)

# Print success message
rich.print(f"✅ {action} complete")
rich.print(f" [dim]Organization:[/dim] {session.config.organization_name}")
rich.print(f" [dim]Repository:[/dim] {session.config.repo_name}")

# Show repo info from config.toml
config_path = codegen_dir / "config.toml"
if config_path.exists():
config = toml.load(config_path)
rich.print(f" [dim]Organization:[/dim] {config.get('organization_name', 'unknown')}")
rich.print(f" [dim]Repository:[/dim] {config.get('repo_name', 'unknown')}")
rich.print("")
rich.print(get_success_message(codegen_dir, docs_dir, examples_dir))

Expand Down
6 changes: 3 additions & 3 deletions src/codegen/cli/commands/init/render.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
def get_success_message(codegen_dir: Path, docs_dir: Path, examples_dir: Path) -> str:
"""Get the success message to display after initialization."""
return """📁 Folders Created:
[dim] Location:[/dim] .codegen-sh
[dim] Docs:[/dim] .codegen-sh/docs
[dim] Examples:[/dim] .codegen-sh/examples"""
[dim] Location:[/dim] .codegen
[dim] Docs:[/dim] .codegen/docs
[dim] Examples:[/dim] .codegen/examples"""
75 changes: 75 additions & 0 deletions src/codegen/cli/commands/notebook/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
import os
import subprocess
from pathlib import Path

import rich_click as click

from codegen.cli.auth.constants import CODEGEN_DIR
from codegen.cli.auth.decorators import requires_auth
from codegen.cli.auth.session import CodegenSession
from codegen.cli.rich.spinners import create_spinner
from codegen.cli.workspace.decorators import requires_init
from codegen.cli.workspace.venv_manager import VenvManager


def create_jupyter_dir() -> Path:
"""Create and return the jupyter directory."""
jupyter_dir = Path.cwd() / CODEGEN_DIR / "jupyter"
jupyter_dir.mkdir(parents=True, exist_ok=True)
return jupyter_dir


def create_notebook(jupyter_dir: Path) -> Path:
"""Create a new Jupyter notebook if it doesn't exist."""
notebook_path = jupyter_dir / "tmp.ipynb"
if not notebook_path.exists():
notebook_content = {
"cells": [
{
"cell_type": "code",
"execution_count": None,
"metadata": {},
"outputs": [],
"source": ["from codegen import Codebase\n", "\n", "# Initialize codebase\n", "codebase = Codebase('../../')\n"],
}
],
"metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}},
"nbformat": 4,
"nbformat_minor": 4,
}
import json

notebook_path.write_text(json.dumps(notebook_content, indent=2))
return notebook_path


@click.command(name="notebook")
@click.option("--background", is_flag=True, help="Run Jupyter Lab in the background")
@requires_auth
@requires_init
def notebook_command(session: CodegenSession, background: bool = False):
"""Open a Jupyter notebook with the current codebase loaded."""
with create_spinner("Setting up Jupyter environment...") as status:
venv = VenvManager()

if not venv.is_initialized():
status.update("Creating virtual environment...")
venv.create_venv()

status.update("Installing required packages...")
venv.install_packages("codegen", "jupyterlab")

jupyter_dir = create_jupyter_dir()
notebook_path = create_notebook(jupyter_dir)

status.update("Starting Jupyter Lab...")

# Prepare the environment with the virtual environment activated
env = {**os.environ, "VIRTUAL_ENV": str(venv.venv_dir), "PATH": f"{venv.venv_dir}/bin:{os.environ['PATH']}"}

# Start Jupyter Lab
if background:
subprocess.Popen(["jupyter", "lab", str(notebook_path)], env=env, start_new_session=True)
else:
# Run in foreground
subprocess.run(["jupyter", "lab", str(notebook_path)], env=env, check=True)
2 changes: 1 addition & 1 deletion src/codegen/cli/utils/codemod_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def get_decorated(cls, start_path: Path | None = None) -> builtins.list[Decorate
".ruff_cache",
".coverage",
"htmlcov",
".codegen-sh",
".codegen",
}

all_functions = []
Expand Down
Loading
Loading