Skip to content

AI impact dashboard #808

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 12, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions codegen-examples/examples/ai_impact_analysis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# AI Impact Analysis

This script analyzes a codebase to measure and report the impact of AI-generated code contributions. It provides detailed insights about AI vs human contributions, helping teams understand the role of AI in their development process.

## Features

- **Repository Analysis**: Automatically detects and analyzes git repositories:

- Uses current directory if it's a git repo

- Searches parent directories for a git repo

- Falls back to cloning a specified repository if needed

```python
# Basic repository setup
repo_path = os.getcwd()
repo_config = RepoConfig.from_repo_path(repo_path)
repo_operator = RepoOperator(repo_config=repo_config)
project = ProjectConfig.from_repo_operator(repo_operator=repo_operator, programming_language=ProgrammingLanguage.PYTHON)
codebase = Codebase(projects=[project])
```

- **Comprehensive Statistics**:

- Total number of commits and AI vs human contribution percentages
- Files with significant AI contribution (>50%)
- AI-touched symbols and their impact
- Detailed contributor breakdown (human and AI contributors)

```python
# Run the analysis
ai_authors = ["github-actions[bot]", "dependabot[bot]"]
results = analyze_ai_impact(codebase, ai_authors)

# Access statistics
stats = results["stats"]
print(f"Total commits: {stats['total_commits']}")
print(f"AI commits: {stats['ai_commits']} ({stats['ai_percentage']:.1f}%)")
print(f"Files with >50% AI: {stats['ai_file_count']} of {stats['total_file_count']}")

# View contributors
for author, count in results["contributors"]:
is_ai = any(ai_name in author for ai_name in ai_authors)
print(f"{'🤖' if is_ai else '👤'} {author}: {count} commits")
```

- **High-Impact Code Detection**:

- Identifies AI-written code that is heavily used by other parts of the codebase
- Shows dependency relationships for AI-contributed code

```python
# Access high-impact AI symbols
for symbol in results["high_impact_symbols"]:
print(f"Symbol: {symbol['name']} ({symbol['filepath']})")
print(f"Used by {symbol['usage_count']} other symbols")
print(f"Last edited by: {symbol['last_editor']}")

# View top AI-contributed files
for file_path, percentage in stats["top_ai_files"]:
print(f"{file_path}: {percentage:.1f}% AI contribution")
```

- **Detailed Attribution**:

- Maps symbols to git history
- Tracks last editor and complete editor history for each symbol
- Flags AI-authored symbols

```python
# Get attribution information for a specific symbol
symbol = codebase.get_symbol("path/to/file.py:MyClass.my_method")

# Access attribution data
print(f"Last editor: {symbol.last_editor}")
print(f"Editor history: {symbol.editor_history}")
print(f"AI authored: {symbol.is_ai_authored}")

# Find all AI-authored symbols
ai_symbols = [s for s in codebase.get_symbols() if s.is_ai_authored]
for symbol in ai_symbols:
print(f"AI symbol: {symbol.name}")
```

## Output

The script generates:

1. Console output with summary statistics
1. Detailed analysis in `ai_impact_analysis.json`
1. Attribution information added to codebase symbols

## Usage

```bash
python run.py
```

The script will automatically:

1. Initialize and analyze the codebase
1. Process git history
1. Generate attribution information
1. Output detailed statistics

You can also visualize the AI impact analysis results using a dashboard. For setup and usage instructions, please see the documentation in the `/dashboard` subdirectory.

## Symbol Attribution

After running the analysis, symbols in the codebase will have the following attribution information:

- `symbol.last_editor`: The last person who edited the symbol
- `symbol.editor_history`: List of all editors who have touched the symbol
- `symbol.is_ai_authored`: Boolean indicating if the symbol was authored by AI

## Learn More

- [Attributions](https://docs.codegen.com/tutorials/attributions)
- [Codegen Documentation](https://docs.codegen.com)

## Contributing

Feel free to submit issues and enhancement requests!
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# AI Impact Analysis Dashboard

A web dashboard for visualizing AI-generated code contributions in your codebase. This dashboard provides detailed insights about AI vs human contributions, helping understand the role of AI in a codebase development process.

## Setup

### Backend

1. Install dependencies:

```bash
uv venv
source .venv/bin/activate
uv pip install modal codegen fastapi
```

2. Deploy or serve the Modal endpoint:

```bash
modal serve backend/api.py
```

```bash
modal deploy backend/api.py
```

### Frontend

1. Install dependencies:

```bash
cd frontend
npm install
```

2. Update the API endpoint:
Edit the fetch URL on line 29 in `components/repo-analysis-dashboard.tsx` to point to your Modal endpoint:

```bash
fetch(`[your-modal-deployment-url]/analyze?repo_full_name=${repoFullName}`, {
method: 'POST',
})
```

3. Start the development server:

```bash
npm run dev
```

## Usage

1. Visit the dashboard in your browser (default: http://localhost:3000)
1. Enter a GitHub repository name (format: username/repo)
1. Click "Analyze Repo" to generate insights

The dashboard will display:

- Summary statistics of AI contributions
- Monthly contribution timeline
- Top files with AI contributions
- High-impact AI-authored symbols
- Contributor breakdown visualization

## Architecture

- **Backend**: Modal-deployed FastAPI service that:

- Clones and analyzes repositories
- Processes git history
- Calculates AI impact metrics
- Returns structured analysis data

- **Frontend**: Next.js application with:

- Interactive charts
- Visualized AI impact metrics

## Learn More

- [AI Impact Analysis Documentation](https://docs.codegen.com/tutorials/attributions)
- [Codegen Documentation](https://docs.codegen.com)

## Contributing

Feel free to submit issues and enhancement requests!
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
from codegen import Codebase
from codegen.extensions.attribution.main import (
add_attribution_to_symbols,
analyze_ai_impact,
)
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
import modal

image = modal.Image.debian_slim().apt_install("git").pip_install("codegen", "fastapi", "intervaltree", "pygit2", "requests")

app = modal.App(name="ai-impact-analysis", image=image)

fastapi_app = FastAPI()

fastapi_app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)


@fastapi_app.post("/analyze")
async def analyze(repo_full_name: str):
codebase = Codebase.from_repo(repo_full_name=repo_full_name, language="python", full_history=True)

print("🤖 Analyzing AI impact on codebase...")

ai_authors = [
"renovate[bot]",
"dependabot[bot]",
"github-actions[bot]",
"devin-ai-integration[bot]",
]

results = analyze_ai_impact(codebase, ai_authors)

print("\n🏷️ Adding attribution information to symbols...")
add_attribution_to_symbols(codebase, ai_authors)
print("✅ Attribution information added to symbols")

return results


@app.function(image=image)
@modal.asgi_app()
def fastapi_modal_app():
return fastapi_app


if __name__ == "__main__":
app.deploy("ai-impact-analysis")
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
@tailwind base;
@tailwind components;
@tailwind utilities;

@layer base {
:root {
--background: 0 0% 100%;
--foreground: 222.2 84% 4.9%;

--card: 0 0% 100%;
--card-foreground: 222.2 84% 4.9%;

--popover: 0 0% 100%;
--popover-foreground: 222.2 84% 4.9%;

--primary: 221.2 83.2% 53.3%;
--primary-foreground: 210 40% 98%;

--secondary: 210 40% 96.1%;
--secondary-foreground: 222.2 47.4% 11.2%;

--muted: 210 40% 96.1%;
--muted-foreground: 215.4 16.3% 46.9%;

--accent: 210 40% 96.1%;
--accent-foreground: 222.2 47.4% 11.2%;

--destructive: 0 84.2% 60.2%;
--destructive-foreground: 210 40% 98%;

--border: 214.3 31.8% 91.4%;
--input: 214.3 31.8% 91.4%;
--ring: 221.2 83.2% 53.3%;

--radius: 0.5rem;
}

.dark {
--background: 222.2 84% 4.9%;
--foreground: 210 40% 98%;

--card: 222.2 84% 4.9%;
--card-foreground: 210 40% 98%;

--popover: 222.2 84% 4.9%;
--popover-foreground: 210 40% 98%;

--primary: 217.2 91.2% 59.8%;
--primary-foreground: 222.2 47.4% 11.2%;

--secondary: 217.2 32.6% 17.5%;
--secondary-foreground: 210 40% 98%;

--muted: 217.2 32.6% 17.5%;
--muted-foreground: 215 20.2% 65.1%;

--accent: 217.2 32.6% 17.5%;
--accent-foreground: 210 40% 98%;

--destructive: 0 62.8% 30.6%;
--destructive-foreground: 210 40% 98%;

--border: 217.2 32.6% 17.5%;
--input: 217.2 32.6% 17.5%;
--ring: 224.3 76.3% 48%;
}
}

@layer base {
* {
@apply border-border;
}
body {
@apply bg-background text-foreground;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import type { Metadata } from "next";
import { Inter } from "next/font/google";
import type React from "react";
import "./globals.css";
import { ThemeProvider } from "@/components/theme-provider";

const inter = Inter({ subsets: ["latin"] });

export const metadata: Metadata = {
title: "AI Code Impact Analysis",
};

export default function RootLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
<html lang="en" suppressHydrationWarning>
<body className={inter.className}>
<ThemeProvider
attribute="class"
defaultTheme="light"
enableSystem
disableTransitionOnChange
>
{children}
</ThemeProvider>
</body>
</html>
);
}

import "./globals.css";
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { RepoAnalysisDashboard } from "@/components/repo-analysis-dashboard";

export default function Home() {
return (
<main className="min-h-screen bg-background">
<RepoAnalysisDashboard />
</main>
);
}
Loading
Loading