cjm-nbdev-docments

Automated documentation compliance checking and fixing for nbdev projects using fastcore.docments style

Install

pip install cjm-nbdev-docments

Overview

cjm-nbdev-docments helps nbdev users adopt and maintain the fastcore.docments documentation style. Instead of traditional docstrings, docments uses inline parameter documentation, making code more concise and readable.

What is docments style?

Instead of this:

def add(x, y):
    """Add two numbers.
    
    Args:
        x: First number
        y: Second number
        
    Returns:
        Sum of x and y
    """
    return x + y

Docments style looks like this:

def add(
    x: int,  # First number
    y: int   # Second number  
) -> int:  # Sum of x and y
    "Add two numbers"
    return x + y

Key Features

🔍 Comprehensive Scanning: Automatically scans all exported functions and classes in your nbdev notebooks
✅ Compliance Checking: Verifies that all parameters and return values have proper documentation
📊 Detailed Reports: Generate text or JSON reports showing compliance status
🔧 Auto-fix Support: Automatically add TODO placeholders for missing documentation
🔄 Docstring Conversion: Convert existing Google/NumPy/Sphinx style docstrings to docments format
💻 CLI Interface: Easy-to-use command-line tool integrated with nbdev workflow

Installation

Install latest from the GitHub repository:

$ pip install git+https://github.com/cj-mills/cjm-nbdev-docments.git

or from conda

$ conda install -c cj-mills cjm_nbdev_docments

or from pypi

$ pip install cjm_nbdev_docments

Quick Start

Basic Usage

Check your nbdev project for documentation compliance:

# Check all notebooks in your project
nbdev-docments

# Get detailed report including compliant functions
nbdev-docments --verbose

# Save report to a file
nbdev-docments --output compliance-report.txt

Auto-fixing Non-compliant Code

Automatically add TODO placeholders for missing documentation:

# Preview what would be fixed
nbdev-docments --fix --dry-run

# Apply fixes
nbdev-docments --fix

Converting Existing Docstrings

Convert traditional docstrings to docments format:

# Convert Google/NumPy/Sphinx style docstrings
nbdev-docments --fix --convert-docstrings

Detailed Usage Examples

Checking a Single Function

You can check individual functions for compliance:

from cjm_nbdev_docments.core import check_function

def example_func(x, y):
    return x + y

check_function(example_func)

Checking a Specific Notebook

Check a single notebook file:

from cjm_nbdev_docments.core import check_notebook

check_notebook("00_core.ipynb")

Programmatic Usage

For integration into your own tools:

from cjm_nbdev_docments.report import check_project, generate_json_report

# Check entire project
results = check_project()

# Generate JSON report
report = generate_json_report(results)

# Process results programmatically
for notebook, data in report['by_notebook'].items():
    print(f"{notebook}: {len(data['non_compliant'])} issues")

What Gets Checked?

The tool checks for:

Function/Method Documentation:
- Presence of a docstring
- Documentation for each parameter (except self)
- Documentation for return values (when return type is annotated)
Type Hints:
- Missing type annotations for parameters
- Missing return type annotations
Class Documentation:
- Presence of class docstrings
TODO Tracking:
- Identifies documentation with TODO placeholders
- Helps track documentation debt

Project Structure

nbs/
├── autofix.ipynb # Automatically add placeholder documentation to non-compliant functions
├── cli.ipynb     # Command-line interface for docments compliance checking
├── core.ipynb    # Core functionality for checking docments compliance
├── report.ipynb  # Generate compliance reports for docments validation
└── scanner.ipynb # Scan nbdev notebooks for exported functions and classes

Total: 5 notebooks

Module Dependencies

graph LR
    autofix[autofix<br/>Auto-Fix]
    cli[cli<br/>CLI Interface]
    core[core<br/>Core]
    report[report<br/>Report Generator]
    scanner[scanner<br/>Scanner]

    autofix --> core
    autofix --> scanner
    cli --> report
    report --> core
    report --> scanner

5 cross-module dependencies detected

CLI Reference

`nbdev-docments` Command

usage: nbdev-docments [-h] [--nbs-path NBS_PATH] [--format {text,json}]
                      [--output OUTPUT] [--verbose] [--quiet] [--todos-only]
                      [--fix] [--convert-docstrings] [--dry-run]

Check nbdev project for docments compliance

options:
  -h, --help            show this help message and exit
  --nbs-path NBS_PATH   Path to notebooks directory (defaults to nbdev config)
  --format {text,json}  Output format (default: text)
  --output OUTPUT, -o OUTPUT
                        Save report to file instead of printing
  --verbose, -v         Show compliant definitions in text report
  --quiet, -q           Only show summary (exit code indicates compliance)
  --todos-only          Show only functions with TODO placeholders
  --fix                 Auto-fix non-compliant functions by adding placeholder
                        docs
  --convert-docstrings  Convert existing Google/NumPy/Sphinx docstrings to
                        docments format (use with --fix)
  --dry-run             Show what would be fixed without making changes

Examples:
  # Check current project
  nbdev-docments
  
  # Check specific notebooks directory
  nbdev-docments --nbs-path ./notebooks
  
  # Generate JSON report
  nbdev-docments --format json
  
  # Save report to file
  nbdev-docments --output report.txt
  
  # Show all definitions (including compliant ones)
  nbdev-docments --verbose
  
  # Show only functions with TODO placeholders
  nbdev-docments --todos-only
  
  # Auto-fix non-compliant functions
  nbdev-docments --fix
  
  # Auto-fix with docstring conversion
  nbdev-docments --fix --convert-docstrings
  
  # Preview fixes without applying
  nbdev-docments --fix --dry-run

For detailed help on any command, use nbdev-docments <command> --help.

Module Overview

Detailed documentation for each module in the project:

Auto-Fix (`autofix.ipynb`)

Automatically add placeholder documentation to non-compliant functions

Import

from cjm_nbdev_docments.autofix import (
    find_signature_boundaries,
    split_parameters,
    parse_single_line_signature,
    generate_param_todo_comment,
    generate_return_todo_comment,
    build_fixed_single_line_function,
    fix_multi_line_signature,
    fix_class_definition,
    insert_function_docstring,
    fix_single_line_function,
    fix_multi_line_function,
    generate_fixed_source,
    fix_notebook,
    DocstringInfo,
    detect_docstring_style,
    parse_google_docstring,
    parse_numpy_docstring,
    parse_sphinx_docstring,
    extract_docstring_info,
    convert_to_docments_format,
    convert_single_line_to_docments,
    convert_multiline_to_docments,
    replace_docstring_in_body,
    generate_fixed_source_with_conversion,
    fix_notebook_with_conversion
)

Functions

@patch
def needs_fixing(
    self: DocmentsCheckResult
) -> bool:  # Whether the definition needs fixing
    "Check if this definition needs any fixing"

@patch
def get_param_name(
    self: DocmentsCheckResult,
    param_str: str  # Parameter string (e.g., "x: int" or "y=10")
) -> str:  # Extracted parameter name
    "Extract parameter name from a parameter string"

@patch
def needs_param_fix(
    self: DocmentsCheckResult,
    param_name: str  # Name of the parameter to check
) -> bool:  # Whether the parameter needs fixing
    "Check if a parameter needs documentation or type hint fixes"

def find_signature_boundaries(
    lines: List[str]  # Source code lines
) -> tuple[int, int]:  # (def_line_idx, sig_end_idx) or (-1, -1) if not found
    "Find the start and end lines of a function signature"

def split_parameters(
    params_str: str  # Parameter string from function signature
) -> List[str]:  # List of individual parameter strings
    "Split a parameter string into individual parameters, handling nested types"

def parse_single_line_signature(
    sig_line: str  # Single-line function signature
) -> dict:  # Parsed components of the signature
    "Parse a single-line function signature into its components"

def generate_param_todo_comment(
    param_name: str,  # Parameter name
    result: DocmentsCheckResult,  # Check result with type hint and doc info
    existing_comment: str = ""  # Existing comment text (without #)
) -> str:  # TODO comment to add
    "Generate appropriate TODO comment for a parameter based on what's missing"

def generate_return_todo_comment(
    result: DocmentsCheckResult,  # Check result with type hint and doc info
    existing_comment: str = ""  # Existing comment text (without #)
) -> str:  # TODO comment to add
    "Generate appropriate TODO comment for return value based on what's missing"

def build_fixed_single_line_function(
    parsed: dict,  # Parsed signature components
    params: List[str],  # Individual parameter strings
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Lines of fixed function signature
    "Build a fixed single-line function with documentation comments"

def fix_multi_line_signature(
    lines: List[str],  # All source lines
    def_line_idx: int,  # Start of function definition
    sig_end_idx: int,  # End of function signature
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Fixed lines for the signature portion
    "Fix a multi-line function signature by adding parameter comments"

def fix_class_definition(
    result: DocmentsCheckResult  # Check result with non-compliant class
) -> str:  # Fixed source code with class docstring
    "Fix a class definition by adding a docstring if missing"

def insert_function_docstring(
    lines: List[str],  # Fixed function lines
    def_line_idx: int,  # Index of function definition line
    indent: str  # Base indentation for the function
) -> List[str]:  # Lines with docstring inserted
    "Insert a TODO docstring after the function signature"

def fix_single_line_function(
    lines: List[str],  # All source lines
    def_line_idx: int,  # Index of function definition line
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Fixed lines for the function
    "Fix a single-line function signature by converting to multi-line with parameter comments"

def fix_multi_line_function(
    lines: List[str],  # All source lines
    def_line_idx: int,  # Start of function definition
    sig_end_idx: int,  # End of function signature
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Fixed lines for the function
    "Fix a multi-line function signature by adding parameter comments"

def generate_fixed_source(
    result: DocmentsCheckResult  # Check result with non-compliant function
) -> str:  # Fixed source code with placeholder documentation
    "Generate fixed source code for a non-compliant function or class"

def fix_notebook(
    nb_path: Path,  # Path to notebook to fix
    dry_run: bool = False  # If True, show changes without saving
) -> Dict[str, Any]:  # Summary of changes made
    "Fix non-compliant functions in a notebook by adding placeholder documentation"

def detect_docstring_style(
    docstring: str  # Docstring text to analyze
) -> str:  # Detected style: 'google', 'numpy', 'sphinx', 'docments', or 'unknown'
    "Detect the style of a docstring"

def parse_google_docstring(
    docstring: str  # Google-style docstring text
) -> DocstringInfo:  # Parsed docstring information
    "Parse a Google-style docstring"

def parse_numpy_docstring(
    docstring: str  # NumPy-style docstring text
) -> DocstringInfo:  # Parsed docstring information
    "Parse a NumPy-style docstring"

def parse_sphinx_docstring(
    docstring: str  # Sphinx-style docstring text
) -> DocstringInfo:  # Parsed docstring information
    "Parse a Sphinx-style docstring"

def extract_docstring_info(
    source: str,  # Function source code
    name: str  # Function name
) -> Optional[DocstringInfo]:  # Extracted docstring information or None
    "Extract docstring information from function source code"

def convert_to_docments_format(
    source: str,  # Original function source code
    docstring_info: DocstringInfo,  # Extracted docstring information
    result: DocmentsCheckResult  # Check result with missing params info
) -> str:  # Converted source code in docments format
    "Convert function source to docments format using extracted docstring info"

def convert_single_line_to_docments(
    sig_line: str,  # Single-line function signature
    docstring_info: DocstringInfo,  # Extracted docstring information
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Multi-line signature with docments comments
    "Convert single-line function signature to multi-line docments format"

def convert_multiline_to_docments(
    sig_lines: List[str],  # Multi-line function signature
    docstring_info: DocstringInfo,  # Extracted docstring information
    result: DocmentsCheckResult  # Check result with missing params info
) -> List[str]:  # Multi-line signature with docments comments
    "Convert multi-line function signature to docments format"

def replace_docstring_in_body(
    body_lines: List[str],  # Function body lines
    description: str,  # New description to use
    def_line: str  # Function definition line for indentation
) -> List[str]:  # Modified body lines
    "Replace the docstring in function body with a simple description"

def generate_fixed_source_with_conversion(
    result: DocmentsCheckResult  # Check result with non-compliant function
) -> str:  # Fixed source code with converted documentation
    "Generate fixed source code, converting existing docstrings to docments format if possible"

def fix_notebook_with_conversion(
    nb_path: Path,  # Path to notebook to fix
    dry_run: bool = False,  # If True, show changes without saving
    convert_docstrings: bool = True  # If True, convert existing docstrings to docments format
) -> Dict[str, Any]:  # Summary of changes made
    "Fix non-compliant functions in a notebook, optionally converting docstrings to docments format"

Classes

class DocstringInfo(NamedTuple):
    "Information extracted from a docstring"

CLI Interface (`cli.ipynb`)

Command-line interface for docments compliance checking

Import

from cjm_nbdev_docments.cli import (
    create_parser,
    handle_autofix,
    generate_report,
    output_report,
    main
)

Functions

def create_parser(
) -> argparse.ArgumentParser:  # Configured argument parser
    "Create and configure the argument parser for docments CLI"

def handle_autofix(
    args: argparse.Namespace  # Parsed command line arguments
) -> int:  # Exit code
    "Handle auto-fix mode for non-compliant functions"

def generate_report(
    results: list,  # Check results from check_project
    format: str,  # Output format ("text" or "json")
    verbose: bool = False  # Whether to show compliant definitions
) -> str:  # Generated report as string
    "Generate a report in the specified format"

def output_report(
    report: str,  # Report content to output
    output_path: Optional[Path] = None,  # File path to save report to
    quiet: bool = False  # Whether to suppress output
) -> None
    "Output the report to console or file"

def main(
    args: Optional[list] = None  # Command line arguments (for testing)
) -> int:  # Exit code (0 for success, 1 for non-compliance)
    "Main CLI entry point for docments checker"

Core (`core.ipynb`)

Core functionality for checking docments compliance

Import

from cjm_nbdev_docments.core import (
    DocmentsCheckResult,
    extract_param_docs_from_func,
    extract_param_docs,
    check_return_doc,
    count_todos_in_docs,
    check_has_docstring_from_func,
    check_has_docstring,
    function_has_return_value,
    check_type_hints,
    check_params_documentation,
    determine_compliance,
    check_definition,
    check_notebook,
    check_function
)

Functions

def extract_param_docs_from_func(
    func: Callable    # Function object to extract docs from
) -> Dict[str, str]:  # Mapping of parameter names to their documentation
    "Extract parameter documentation from function object using fastcore.docments"

def extract_param_docs(
    source:str    # Function source code
) -> Dict[str, str]:  # Mapping of parameter names to their documentation
    "Extract parameter documentation from function source using docments style (fallback)"

def check_return_doc(
    source: str  # Function source code
) -> bool:  # Whether return is documented
    "Check if function has return documentation"

def count_todos_in_docs(
    source: str,  # Function/class source code
    name: str  # Name of the function/class for AST parsing
) -> Tuple[int, bool]:  # (todo_count, has_todos)
    "Count TODO placeholders only in documentation (docstring, param docs, return docs)"

def check_has_docstring_from_func(
    func: Callable  # Function object to check
) -> bool:  # Whether the function has a docstring
    "Check if a function has a docstring using fastcore.docments"

def check_has_docstring(
    source: str,  # Function/class source code
    name: str  # Name of the function/class
) -> bool:  # Whether the definition has a docstring
    "Check if a function/class has a docstring using AST parsing (fallback)"

def function_has_return_value(
    source: str,  # Function source code
    name: str  # Function name
) -> bool:  # Whether function has explicit return statements with values
    "Check if a function actually returns a value (not just implicit None)"

def check_type_hints(
    definition: Dict[str, Any],  # Definition dict from scanner
    source: Optional[str] = None  # Function source code (optional)
) -> Tuple[Dict[str, bool], List[str], bool]:  # (params_with_type_hints, missing_type_hints, return_has_type_hint)
    "Check which parameters and return value have type hints"

def check_params_documentation(
    definition: Dict[str, Any],  # Definition dict from scanner
    source: str  # Function source code
) -> Tuple[Dict[str, bool], List[str], bool]:  # (params_documented, missing_params, return_documented)
    "Check parameter and return documentation for a function"

def determine_compliance(
    has_docstring: bool,  # Whether definition has a docstring
    params_documented: Dict[str, bool],  # Which params have documentation
    return_documented: bool  # Whether return is documented
) -> bool:  # Overall compliance status
    "Determine if a definition is compliant based on documentation checks"

def check_definition(
    definition: Dict[str, Any]  # Definition dict from scanner
) -> DocmentsCheckResult:  # Check result with compliance details
    "Check a function/class definition for docments compliance"

def check_notebook(
    nb_path: str  # Path to notebook file  
) -> None:  # Prints compliance report
    "Check a specific notebook for docments compliance"

def check_function(
    func:Callable          # Function object to check
) -> DocmentsCheckResult:  # Check result for the function
    "Check a single function for docments compliance"

Classes

@dataclass
class DocmentsCheckResult:
    "Result of checking a function/class for docments compliance"
    
    name: str  # Name of the function/class
    type: str  # Type (FunctionDef, ClassDef, etc.)
    notebook: str  # Source notebook
    has_docstring: bool  # Whether it has a docstring
    params_documented: Dict[str, bool]  # Which params have documentation
    return_documented: bool  # Whether return is documented
    missing_params: List[str]  # Parameters missing documentation
    is_compliant: bool  # Overall compliance status
    source: str  # Source code of the definition
    has_todos: bool = False  # Whether it contains TODO placeholders
    todo_count: int = 0  # Number of TODO placeholders found
    params_with_type_hints: Dict[str, bool]  # Which params have type hints
    return_has_type_hint: bool = False  # Whether return has type hint
    params_missing_type_hints: List[str]  # Parameters missing type hints

Report Generator (`report.ipynb`)

Generate compliance reports for docments validation

Import

from cjm_nbdev_docments.report import (
    check_project,
    generate_text_report,
    generate_json_report
)

Functions

def check_project(
    nbs_path: Optional[Path] = None  # Path to notebooks directory
) -> List[DocmentsCheckResult]:  # List of check results for all definitions
    "Check all exported definitions in a project for docments compliance"

def _generate_summary_stats(
    results: List[DocmentsCheckResult]  # Check results to summarize
) -> List[str]:  # Lines of summary statistics
    "Generate summary statistics section of the report"

def _generate_non_compliant_section(
    results: List[DocmentsCheckResult],  # Check results
    by_notebook: Dict[str, List[DocmentsCheckResult]]  # Results grouped by notebook
) -> List[str]:  # Lines of non-compliant section
    "Generate non-compliant definitions section of the report"

def _generate_todos_section(
    results: List[DocmentsCheckResult],  # Check results
    by_notebook: Dict[str, List[DocmentsCheckResult]]  # Results grouped by notebook
) -> List[str]:  # Lines of TODOs section
    "Generate TODO placeholders section of the report"

def _generate_compliant_section(
    results: List[DocmentsCheckResult],  # Check results
    by_notebook: Dict[str, List[DocmentsCheckResult]]  # Results grouped by notebook
) -> List[str]:  # Lines of compliant section
    "Generate compliant definitions section of the report"

def generate_text_report(
    results: List[DocmentsCheckResult],  # Check results from check_project
    verbose: bool = False  # Include detailed information
) -> str:  # Formatted text report
    "Generate a human-readable text report of compliance results"

def generate_json_report(
    results: List[DocmentsCheckResult]  # Check results from check_project
) -> Dict[str, Any]:  # JSON-serializable report data
    "Generate a JSON report of compliance results"

Scanner (`scanner.ipynb`)

Scan nbdev notebooks for exported functions and classes

Import

from cjm_nbdev_docments.scanner import (
    get_export_cells,
    extract_definitions,
    scan_notebook,
    scan_project
)

Functions

def get_export_cells(
    nb_path: Path    # Path to the notebook file
) -> List[Dict[str, Any]]:  # List of cells with export directives
    "Extract all code cells from a notebook that have export directives"

def extract_definitions(
    source: str  # Python source code
) -> List[Dict[str, Any]]:  # List of function/class definitions with metadata
    "Extract function and class definitions from source code"

def scan_notebook(
    nb_path: Path,  # Path to the notebook to scan
    nbs_root: Optional[Path] = None  # Root notebooks directory (for relative paths)
) -> List[Dict[str, Any]]:  # List of exported definitions with metadata
    "Scan a notebook and extract all exported function/class definitions"

def scan_project(
    nbs_path: Optional[Path] = None,  # Path to notebooks directory (defaults to config.nbs_path)
    pattern: str = "*.ipynb"  # Pattern for notebook files to scan
) -> List[Dict[str, Any]]:  # All exported definitions found in the project
    "Scan all notebooks in a project for exported definitions"

Install

Overview

What is docments style?

Key Features

Installation

Quick Start

Basic Usage

Auto-fixing Non-compliant Code

Converting Existing Docstrings

Detailed Usage Examples

Checking a Single Function

Checking a Specific Notebook

Programmatic Usage

What Gets Checked?

Project Structure

Module Dependencies

CLI Reference

nbdev-docments Command

Module Overview

Auto-Fix (autofix.ipynb)

Import

Functions

Classes

CLI Interface (cli.ipynb)

Import

Functions

Core (core.ipynb)

Import

Functions

Classes

Report Generator (report.ipynb)

Import

Functions

Scanner (scanner.ipynb)

Import

Functions

`nbdev-docments` Command

Auto-Fix (`autofix.ipynb`)

CLI Interface (`cli.ipynb`)

Core (`core.ipynb`)

Report Generator (`report.ipynb`)

Scanner (`scanner.ipynb`)