Skip to content

[CG-7930] feat: remove unused imports after moving symbol & new api for removing unused symbols #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
23ea9e5
remove imports after move
Jan 23, 2025
833c7a8
Merge remote-tracking branch 'origin/develop' into tom-cg-7930-remove…
Jan 30, 2025
2542923
merge
Jan 30, 2025
abb5bc6
return rmd code
Jan 30, 2025
c65b534
improve python import resolution etc
Jan 30, 2025
7670afa
py test fix
Jan 31, 2025
004cf06
typescript UTs and code changes for new edge cases
Jan 31, 2025
5556510
UT fixes
Feb 4, 2025
daf1b75
Automated pre-commit update
tkfoss Feb 4, 2025
5da1428
rm prints
Feb 4, 2025
57577ff
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
EdwardJXLi Feb 5, 2025
32769df
Fix tests for Python `test_function_move_to_file`
EdwardJXLi Feb 5, 2025
6b7e688
Fix tests for Typescript `test_function_move_to_file`
EdwardJXLi Feb 5, 2025
f744bf2
Remove apidoc from remove_unused_imports and remove_unused_exports
EdwardJXLi Feb 5, 2025
7123cf9
Remove py_apidoc from is_from_import
EdwardJXLi Feb 5, 2025
6a31479
simplify py code
Feb 6, 2025
4aa7676
Feature flag generics support (#304)
bagel897 Feb 5, 2025
682c428
Specify language on 'codegen init' in CLI (#289)
vishalshenoy Feb 5, 2025
f6290cf
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
3c33141
Fix: duplicate edge creation (#305)
bagel897 Feb 5, 2025
62e1176
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
5ef9f27
chore(ci): CG-10672 add back 3.13 mac build (#302)
christinewangcw Feb 5, 2025
624295c
ci: don't report coverage data as json (#307)
bagel897 Feb 5, 2025
b22c368
fix: pre-commit on develop branch (#309)
christinewangcw Feb 5, 2025
fbeb41b
fix: pre-commit missing `--source` (#312)
christinewangcw Feb 5, 2025
956d3ea
docs: Add docs for incremental recomputation (#311)
bagel897 Feb 5, 2025
01f2fa1
chore(deps): update dependency aws-cli to v5.1.4 (#310)
renovate[bot] Feb 5, 2025
b209125
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
6af57d8
chore(deps): update dependency aws-cli to v5.2.0 (#314)
renovate[bot] Feb 5, 2025
c7b4401
docs: add Python 3.13 recommendation to README (#303)
devin-ai-integration[bot] Feb 5, 2025
310891f
chore(ci): [CG-10689] add slack alert in release (#316)
christinewangcw Feb 5, 2025
658b010
Ignore folder (#317)
eacodegen Feb 5, 2025
50d40d9
chore(ci): clean-up circle ci workflows (#319)
christinewangcw Feb 5, 2025
4e0a66e
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
13c9ec4
Set default value (#322)
bagel897 Feb 6, 2025
737e555
Mypyc/cython changes (#318)
bagel897 Feb 6, 2025
b07e6f2
fix bug (#323)
bagel897 Feb 6, 2025
af179b5
fix: empty collection remove (#324)
bagel897 Feb 6, 2025
75dd7dc
fix(ci): invalid gh template (#320)
christinewangcw Feb 6, 2025
589d089
chore(ci): add issue comment for arm + remove install deps (#325)
christinewangcw Feb 6, 2025
2820313
Fix JSX prop parsing (#326)
bagel897 Feb 6, 2025
676ce46
feat(ci) CG-10496: semantic release (#328)
christinewangcw Feb 6, 2025
bf7b7d0
chore: separate workflow for semantic (#329)
christinewangcw Feb 6, 2025
6de94d3
chore(ci): delete circle CI validate hook (#330)
christinewangcw Feb 6, 2025
77ab8b9
chore: add workflow dispatch to auto release (#331)
christinewangcw Feb 6, 2025
cce834c
chore(deps): update pre-commit hook astral-sh/uv-pre-commit to v0.5.2…
renovate[bot] Feb 6, 2025
dc04179
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
9f1b9ae
chore(ci): set build skip in pyproject (#337)
christinewangcw Feb 6, 2025
56eceff
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
48c3ffb
git
Feb 6, 2025
641b0aa
test remove unused imports
frainfreeze Feb 7, 2025
fe36690
Automated pre-commit update
tkfoss Feb 7, 2025
d71fca8
UTs & export refactor
frainfreeze Feb 7, 2025
a225999
Automated pre-commit update
tkfoss Feb 7, 2025
cee50b2
revert github workflow file, py code, ut changes
frainfreeze Feb 7, 2025
8ca64ff
Automated pre-commit update
tkfoss Feb 7, 2025
d2f3414
update
Feb 10, 2025
37664b9
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.9…
renovate[bot] Feb 6, 2025
bfbd7b0
Remove og image default (#345)
joelaguero Feb 6, 2025
ef987af
feat(docs): Changelog generation (#341)
jemeza-codegen Feb 6, 2025
34cdf58
Foundations for PR BOT static analisis (#343)
kopekC Feb 6, 2025
e429498
chore(deps): update dependency httpx to <0.28.2,>=0.28.1 (#346)
renovate[bot] Feb 6, 2025
440a57f
Tawsif fix asyncify promise return type (#348)
tawsifkamal Feb 6, 2025
636034a
CG-10694: Remove lowside + enterprise from codegen.git (#349)
caroljung-cg Feb 6, 2025
609c0b6
fix: Disable uv cache (#351)
caroljung-cg Feb 7, 2025
fdfe835
Fix GithubClient constructor (#352)
caroljung-cg Feb 7, 2025
36cf065
chore: Remove access token from local repo operator (#354)
caroljung-cg Feb 7, 2025
14595fb
Changes to default urls (#357)
kopekC Feb 7, 2025
d4fe603
chore: subdir logging (#356)
christinewangcw Feb 7, 2025
516d9c1
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
2bd1e4a
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
2e0449d
feat: [CG-10632] mcp server (#358)
rushilpatel0 Feb 7, 2025
ba4b842
docs: remove overview iframe (#363)
jayhack Feb 7, 2025
ac2c348
fix init.py bug (#364)
bagel897 Feb 7, 2025
547eef0
Experiment: Override `__class__` for Builtin types to shadow `isinsta…
EdwardJXLi Feb 7, 2025
a03ba1c
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
ee5a854
chore: disable failing parse tests for now (#366)
christinewangcw Feb 7, 2025
2995692
docs: switches main button to be github (#373)
jayhack Feb 8, 2025
db70491
docs: removes code link backticks (#369)
jayhack Feb 8, 2025
4abaae0
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 9, 2025
f55f27c
[WIP] Langchain demo (#374)
jayhack Feb 9, 2025
106111e
feat: adds VectorIndex extension (#378)
jayhack Feb 9, 2025
6224ca6
chore(deps): lock file maintenance (#379)
renovate[bot] Feb 10, 2025
df1e801
[WIP] Modal Demo (#381)
jayhack Feb 10, 2025
38aa188
docs: adds code agent tutorial (#382)
jayhack Feb 10, 2025
fe2a026
chore(deps): lock file maintenance (#384)
renovate[bot] Feb 10, 2025
eb38163
codegen-examples is dead, long live codegen-examples (#375)
kopekC Feb 10, 2025
73b93a6
chore(deps): update dependency jupyterlab to v4.3.5 (#385)
renovate[bot] Feb 10, 2025
1fec4b3
fix(deps): update dependency codegen to v0.5.30 (#387)
renovate[bot] Feb 10, 2025
dd54bf9
docs: langchain + modal examples (#386)
jayhack Feb 10, 2025
e36d0b6
[wip] Modal RAG example (#388)
jayhack Feb 10, 2025
d863ad3
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.9…
renovate[bot] Feb 10, 2025
4e5f74a
fix: remove pre-push hook on auto release (#390)
christinewangcw Feb 10, 2025
2763872
CG-10301: renaming file path bug (#392)
Feb 10, 2025
4f17fba
build: Support x86_64 mac (#393)
eacodegen Feb 10, 2025
85a4331
Fix OSS Parse Tests (#372)
EdwardJXLi Feb 10, 2025
029fcfa
Update plotly requirement from <6.0.0,>=5.24.0 to >=5.24.0,<7.0.0 (#246)
dependabot[bot] Feb 10, 2025
4408ae8
fix: wait for checks semantic release (#395)
christinewangcw Feb 10, 2025
cf24fbd
CG-10731: Add ChainedAttribute.attribute_chain (#383)
tawsifkamal Feb 10, 2025
1332f95
fix CG-9440 clean repo - clears from the default branch (#398)
christinewangcw Feb 10, 2025
3524117
chore(testing): set base dir in op creation (#399)
christinewangcw Feb 10, 2025
ef9881e
CG-10470: Add `config` CLI commands (#391)
caroljung-cg Feb 10, 2025
428b289
Remove LFS from `codegen-sdk` (+ disable `disallowed-words` check) (#…
EdwardJXLi Feb 10, 2025
37eb5bb
docs: remove Apple siliicon (#368)
jayhack Feb 10, 2025
e03e067
ops: disable auto-release (#400)
christinewangcw Feb 10, 2025
2fb7d06
Automated pre-commit update
tkfoss Feb 10, 2025
2642cca
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
Feb 10, 2025
2ef105c
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ruff.toml
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,11 @@ extend-generics = [
"codegen.sdk.python.expressions.named_type.PyNamedType",
"codegen.sdk.python.expressions.string.PyString",
"codegen.sdk.python.expressions.union_type.PyUnionType",
"codegen.sdk.python.file.remove_unused_imports",
"codegen.sdk.python.file.remove_unused_exports",
"codegen.sdk.python.file.PyFile",
"codegen.sdk.python.function.PyFunction",
"codegen.sdk.python.import_resolution.is_from_import",
"codegen.sdk.python.import_resolution.PyImport",
"codegen.sdk.python.interfaces.has_block.PyHasBlock",
"codegen.sdk.python.placeholder.placeholder_return_type.PyReturnTypePlaceholder",
Expand Down
49 changes: 45 additions & 4 deletions src/codegen/sdk/core/symbol.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ def insert_before(self, new_src: str, fix_indentation: bool = False, newline: bo
return first_node.insert_before(new_src, fix_indentation, newline, priority, dedupe)
return super().insert_before(new_src, fix_indentation, newline, priority, dedupe)

def move_to_file(self, file: SourceFile, include_dependencies: bool = True, strategy: str = "update_all_imports") -> None:
def move_to_file(self, file: SourceFile, include_dependencies: bool = True, strategy: str = "update_all_imports", remove_unused_imports: bool = True) -> None:
"""Moves the given symbol to a new file and updates its imports and references.

This method moves a symbol to a new file and updates all references to that symbol throughout the codebase. The way imports are handled can be controlled via the strategy parameter.
Expand All @@ -275,6 +275,7 @@ def move_to_file(self, file: SourceFile, include_dependencies: bool = True, stra
strategy (str): The strategy to use for updating imports. Can be either 'add_back_edge' or 'update_all_imports'. Defaults to 'update_all_imports'.
- 'add_back_edge': Moves the symbol and adds an import in the original file
- 'update_all_imports': Updates all imports and usages of the symbol to reference the new file
remove_unused_imports (bool): If True, removes any imports in the original file that become unused after moving the symbol. Defaults to True.

Returns:
None
Expand All @@ -283,19 +284,25 @@ def move_to_file(self, file: SourceFile, include_dependencies: bool = True, stra
AssertionError: If an invalid strategy is provided.
"""
encountered_symbols = {self}
self._move_to_file(file, encountered_symbols, include_dependencies, strategy)
self._move_to_file(file, encountered_symbols, include_dependencies, strategy, remove_unused_imports)

@noapidoc
def _move_to_file(self, file: SourceFile, encountered_symbols: set[Symbol | Import], include_dependencies: bool = True, strategy: str = "update_all_imports") -> tuple[NodeId, NodeId]:
def _move_to_file(
self, file: SourceFile, encountered_symbols: set[Symbol | Import], include_dependencies: bool = True, strategy: str = "update_all_imports", remove_unused_imports: bool = True
) -> tuple[NodeId, NodeId]:
"""Helper recursive function for `move_to_file`"""
from codegen.sdk.core.import_resolution import Import

# Track original file and imports used by this symbol before moving
symbol_imports = set()

# =====[ Arg checking ]=====
if file == self.file:
return file.file_node_id, self.node_id
if imp := file.get_import(self.name):
encountered_symbols.add(imp)
imp.remove()
if remove_unused_imports:
imp.remove()

if include_dependencies:
# =====[ Move over dependencies recursively ]=====
Expand Down Expand Up @@ -368,6 +375,40 @@ def _move_to_file(self, file: SourceFile, encountered_symbols: set[Symbol | Impo
# =====[ Delete the original symbol ]=====
self.remove()

# After moving a symbol (function or class) out of a file, if there are imports that are now unused because that was the only thing using them, remove those as well
if remove_unused_imports:
# Get all imports that were used by the moved symbol
for dep in self.dependencies:
if isinstance(dep, Import):
symbol_imports.add(dep)

# Check each import that was used by the moved symbol
for import_symbol in symbol_imports:
try:
# Try to access any property - if the import was removed this will fail
_ = import_symbol.file
except (AttributeError, ReferenceError):
# Skip if import was already removed
continue

# Check if import is still used by any remaining symbols
still_used = False
for usage in import_symbol.usages:
# Skip usages from the moved symbol
if usage.usage_symbol == self:
continue

# Skip usages from symbols we moved
if usage.usage_symbol in encountered_symbols:
continue

still_used = True
break

# Remove import if it's no longer used
if not still_used:
import_symbol.remove()

@property
@reader
@noapidoc
Expand Down
69 changes: 66 additions & 3 deletions src/codegen/sdk/python/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,10 +170,73 @@ def add_import_from_import_string(self, import_string: str) -> None:
else:
self.insert_before(import_string, priority=1)

@noapidoc
@py_apidoc
def remove_unused_imports(self) -> None:
"""Removes unused imports from the file.

Handles different Python import styles:
- Single imports (import x)
- From imports (from y import z)
- Multi-imports (from y import (a, b as c))
- Wildcard imports (from x import *)
- Type imports (from typing import List)
- Future imports (from __future__ import annotations)

Preserves:
- Comments and whitespace where possible
- Future imports even if unused
- Type hints and annotations
"""
# Track processed imports to avoid duplicates
processed_imports = set()

# Group imports by module for more efficient processing
module_imports = {}

# First pass - group imports by module
for import_stmt in self.imports:
if import_stmt in processed_imports:
continue

# Always preserve __future__ and star imports since we can't track their usage
print(f"import_stmt: {import_stmt}", import_stmt.is_future_import, import_stmt.is_star_import)
if import_stmt.is_future_import or import_stmt.is_star_import:
continue

module = import_stmt.module_name
if module not in module_imports:
module_imports[module] = []
print(f"Adding {module} to module_imports")
module_imports[module].append(import_stmt)

print(f"module_imports: {module_imports}")
# Second pass - process each module's imports
for module, imports in module_imports.items():
# Skip if any import from this module is used
if any(imp.usages for imp in imports):
# Remove individual unused imports if it's a from-style import
if len(imports) > 1 and imports[0].is_from_import():
for imp in imports:
if not imp.usages and imp not in processed_imports:
processed_imports.add(imp)
imp.remove()
continue

# If no imports from module are used, remove them all
for imp in imports:
if imp not in processed_imports:
processed_imports.add(imp)
imp.remove()

self.G.commit_transactions()

@py_apidoc
def remove_unused_exports(self) -> None:
"""Removes unused exports from the file. NO-OP for python"""
pass
"""Removes unused exports from the file.
In Python this is equivalent to removing unused imports since Python doesn't have
explicit export statements. Calls remove_unused_imports() internally.
"""
self.remove_unused_imports()

@cached_property
@noapidoc
Expand Down
64 changes: 64 additions & 0 deletions src/codegen/sdk/python/import_resolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,70 @@ def get_import_string(
else:
return f"from {import_module} import {self.name}"

@property
def module_name(self) -> str:
"""Gets the module name for this import.

For 'import x' returns 'x'
For 'from x import y' returns 'x'
For 'from .x import y' returns '.x'

Returns:
str: The module name for this import.
"""
if self.ts_node.type == "import_from_statement":
module_node = self.ts_node.child_by_field_name("module_name")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rely on self.module?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean use self.module instead of the .module_name? it seems to miss some imports then. or should module_name itself be changed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's missing imports, that may be a bug in the SDK. Generally we want to use our nodes rather than tree-sitters post-parse

return module_node.text.decode("utf-8") if module_node else ""
return self.ts_node.child_by_field_name("name").text.decode("utf-8")

@py_apidoc
def is_from_import(self) -> bool:
"""Determines if this is a from-style import statement.

Checks if the import uses 'from' syntax (e.g., 'from module import symbol')
rather than direct import syntax (e.g., 'import module').

Returns True for imports like:
- from x import y
- from .x import y
- from x import (a, b, c)
- from x import *

Returns False for:
- import x
- import x as y

Returns:
bool: True if this is a from-style import, False otherwise.
"""
return self.ts_node.type == "import_from_statement"

@property
def is_star_import(self) -> bool:
"""Determines if this is a star import (from x import *).

Returns:
bool: True if this is a star import, False otherwise
"""
if self.ts_node.type != "import_from_statement":
return False

# Look for wildcard_import node among children
wildcard_import = next((node for node in self.ts_node.children if node.type == "wildcard_import"), None)
return wildcard_import is not None

@property
def is_future_import(self) -> bool:
"""Determines if this is a __future__ import.

Returns True for imports like:
- from __future__ import annotations

Returns:
bool: True if this is a __future__ import, False otherwise
"""
return self.ts_node.type == "future_import_statement"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use import_type?



class PyExternalImportResolver(ExternalImportResolver):
def __init__(self, from_alias: str, to_context: CodebaseGraph) -> None:
Expand Down
66 changes: 66 additions & 0 deletions src/codegen/sdk/typescript/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -389,11 +389,27 @@ def get_import_string(self, alias: str | None = None, module: str | None = None,
def valid_import_names(self) -> dict[str, Symbol | TSImport]:
"""Returns a dict mapping name => Symbol (or import) in this file that can be imported from another file"""
valid_export_names = {}

# Handle default exports
if len(self.default_exports) == 1:
valid_export_names["default"] = self.default_exports[0]

# Handle named exports and their aliases
for export in self.exports:
for name, dest in export.names:
# Track both original name and alias if present
valid_export_names[name] = dest
if hasattr(dest, "alias") and dest.alias:
valid_export_names[dest.alias] = dest

# # Handle imports and their aliases
# for import_stmt in self.imports:
# for name, symbol in import_stmt.imported_symbols.items():
# valid_export_names[name] = symbol
# # Also track the alias if present
# if hasattr(symbol, "alias") and symbol.alias:
# valid_export_names[symbol.alias] = symbol

return valid_export_names

####################################################################################################################
Expand Down Expand Up @@ -440,3 +456,53 @@ def get_namespace(self, name: str) -> TSNamespace | None:
TSNamespace | None: The namespace with the specified name if found, None otherwise.
"""
return next((x for x in self.symbols if isinstance(x, TSNamespace) and x.name == name), None)

@writer
def remove_unused_imports(self, moved_symbol_names: set[str] | None = None) -> None:
"""Removes unused imports from the file.

Args:
moved_symbol_names: Optional set of symbol names that were moved to another file
"""
for import_statement in self.import_statements:
# Track which symbols in this import statement are still used
used_symbols = []
removed_symbols = []

for import_symbol in import_statement.imports:
# Skip side effect imports
if import_symbol.import_type == ImportType.SIDE_EFFECT:
continue

symbol_name = import_symbol.alias.source if import_symbol.alias else import_symbol.name

# Check if this import is still used in the file
is_used = False
for usage in import_symbol.usages:
# Skip usages from moved symbols if provided
if moved_symbol_names and usage.usage_symbol and usage.usage_symbol.name in moved_symbol_names:
continue
is_used = True
break

if is_used:
used_symbols.append(import_symbol)
else:
removed_symbols.append(import_symbol)

if not used_symbols and removed_symbols:
# If no symbols are used, remove the entire import statement
import_statement.remove()
elif removed_symbols and used_symbols:
# If some symbols are used but others aren't, update the import statement
new_imports = []
for symbol in used_symbols:
if symbol.alias:
new_imports.append(f"{symbol.name} as {symbol.alias.source}")
else:
new_imports.append(symbol.name)

module = import_statement.module.source
type_prefix = "type " if import_statement.is_type_import else ""
new_statement = f"import {type_prefix}{{ {', '.join(new_imports)} }} from {module};"
import_statement.source = new_statement
Loading
Loading