From 43b15c68da86c57a5cc018e4c93fbc369ba3398e Mon Sep 17 00:00:00 2001 From: Sarthak Agarwal Date: Sat, 9 Nov 2024 19:53:39 +0530 Subject: [PATCH] add secret scanner and monorepo hook (#1201) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### **User description** ### To Test this PR## 1. Clone and switch to the branch and Run `npm install` 2. Fork into a new branch to avoid pushing to this open PR 3. Add a file anywhere like test_secret.py and below code ``` def test_function(): # This is a fake API key for testing purposes api_key = "12345-ABCDE-67890-FGHIJ" print("Testing secret detection.") ```` 4. Add and try commiting the changes ### **User description** closes # https://linear.app/codeflash-ai/issue/CF-442/implement-a-secrets-scanning-solution-pre-commit-on-pr-deployments ___ ### **PR Type** enhancement, configuration changes ___ ### **Description** - Added Secretlint configuration to scan for secrets using recommended rules. - Introduced a shell script to run Ruff linting for Python projects. - Updated README with setup instructions and prerequisites for Node.js, npm, Python, and Mamba. - Configured Lefthook for pre-commit hooks to perform secret scanning and code linting. - Created `package.json` with scripts for Lefthook installation and Secretlint execution, and added relevant development dependencies. ___ ### **Changes walkthrough** 📝
Relevant files
Configuration changes
secretlint.config.js
Add Secretlint configuration with recommended rules           
secretlint.config.js
  • Added configuration for Secretlint.
  • Included a rule preset for recommended secret scanning.
    		•  +7/-0     
    lefthook.yml
    Configure Lefthook for pre-commit secret and code linting
    lefthook.yml
  • Added Lefthook configuration for pre-commit hooks.
  • Configured secret scanning, Python linting, and JS linting commands.
    		•  +20/-0   
    package.json
    Add package.json with scripts and dev dependencies             
    package.json
  • Added project metadata and scripts for Lefthook and Secretlint.
  • Included development dependencies for Lefthook, Secretlint, and
    ESLint.
    		•  +23/-0   
    Enhancement
    run_ruff.sh
    Add shell script for Ruff linting in Python projects         
    run_ruff.sh
  • Added a shell script to run Ruff linting.
  • Configured environment activation and linting command.
    		•  +12/-0   
    Documentation
    README.md
    Update README with project setup and prerequisites             
    README.md
  • Updated project setup instructions.
  • Added prerequisites for Node.js, npm, Python, and Mamba.
    		•  +8/-0     
    ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information ___ ### **PR Type** Enhancement, Configuration changes ___ ### **Description** - Added Secretlint configuration to scan for secrets using recommended rules. - Introduced a shell script to run Ruff linting for Python projects. - Updated README with setup instructions and prerequisites for Node.js, npm, Python, and Mamba. - Configured Lefthook for pre-commit hooks to perform secret scanning and code linting. - Created `package.json` with scripts for Lefthook installation and Secretlint execution, and added relevant development dependencies. ___ ### **Changes walkthrough** 📝
    Relevant files
    Configuration changes
    secretlint.config.js
    Add Secretlint configuration for secret scanning                 
    secretlint.config.js
  • Added Secretlint configuration file.
  • Configured with recommended rules for secret scanning.
    		•  +7/-0     
    lefthook.yml
    Configure Lefthook for pre-commit secret scanning and linting
    lefthook.yml
  • Configured Lefthook for pre-commit hooks.
  • Added commands for secret scanning and code linting.
    		•  +20/-0   
    package.json
    Add package.json with scripts and dependencies                     
    package.json
  • Created package.json for the monorepo.
  • Added scripts for Lefthook and Secretlint.
  • Included development dependencies.
    		•  +23/-0   
    Enhancement
    run_ruff.sh
    Introduce shell script for Ruff linting in Python               
    run_ruff.sh
  • Created a shell script to run Ruff linting.
  • Script navigates to the Python project directory.
  • Activates Mamba environment for linting.
    		•  +12/-0   
    Documentation
    README.md
    Update README with setup instructions and prerequisites   
    README.md
  • Added project setup instructions.
  • Listed prerequisites for Node.js, npm, Python, and Mamba.
    		•  +8/-0     
    ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information --- .gitignore | 8 ++++++++ README.md | 8 ++++++++ lefthook.yml | 14 ++++++++++++++ package.json | 23 +++++++++++++++++++++++ secretlint.config.js | 7 +++++++ 5 files changed, 60 insertions(+) create mode 100644 lefthook.yml create mode 100644 package.json create mode 100644 secretlint.config.js diff --git a/.gitignore b/.gitignore index cc7bf6bcd..fd2c8c696 100644 --- a/.gitignore +++ b/.gitignore @@ -163,5 +163,13 @@ cython_debug/ #.idea/ .aider* /js/common/node_modules/ +/node_modules/ *.xml *.pem + +# Ruff cache +.ruff_cache/ + +# IDE settings +.idea/ +.vscode/ diff --git a/README.md b/README.md index 65b94bd22..7f81fe7db 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,14 @@ Here's the projects that are part of the CodeFlash MonoRepo: - CodeFlash NodeJS CF API - /js/cf-api - CodeFlash Webapp - /js/cf-webapp +# Project Setup + +## Prerequisites + +- **Node.js and npm**: Ensure Node.js is installed and npm is set up for installation of pre-commit hook(Lefthook). +- **Python and Mamba**: Ensure Python is installed and Mamba is set up. + +post clone run `npm install` to install all the dependencies at root level. ## Glossary ### Optimization diff --git a/lefthook.yml b/lefthook.yml new file mode 100644 index 000000000..54ec471cd --- /dev/null +++ b/lefthook.yml @@ -0,0 +1,14 @@ +pre-commit: + parallel: true + commands: + secret-scan: + runner: "node" + glob: "*" + exclude: "node_modules/**|venv/**|.venv/**|__pycache__/**|dist/**|build/**" + command: "./node_modules/.bin/secretlint {staged_files} --maskSecrets --config ./secretlint.config.js" + + js-lint: + runner: "node" + glob: "*.js" + exclude: "node_modules/**|build/**|dist/**" + command: "./node_modules/.bin/eslint {staged_files} --ignore-path .gitignore" diff --git a/package.json b/package.json new file mode 100644 index 000000000..14724c867 --- /dev/null +++ b/package.json @@ -0,0 +1,23 @@ +{ + "name": "codeflashai", + "version": "1.0.0", + "description": "Here's the projects that are part of the CodeFlash MonoRepo:", + "main": "index.js", + "directories": { + "doc": "docs" + }, + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1", + "prepare": "lefthook install", + "secretlint": "secretlint \"**/*\" --maskSecrets --config ./secretlint.config.js --ignore \"**/node_modules/**\" \"**/venv/**\" \"**/.venv/**\" \"**/__pycache__/**\"" + }, + "keywords": [], + "author": "", + "license": "ISC", + "devDependencies": { + "@evilmartians/lefthook": "^1.8.2", + "@secretlint/secretlint-rule-preset-recommend": "^9.0.0", + "eslint": "^9.14.0", + "secretlint": "^9.0.0" + } +} diff --git a/secretlint.config.js b/secretlint.config.js new file mode 100644 index 000000000..3ebbe4fbe --- /dev/null +++ b/secretlint.config.js @@ -0,0 +1,7 @@ +module.exports = { + rules: [ + { + id: "@secretlint/secretlint-rule-preset-recommend", + }, + ], +};