diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..8ecc1cd --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,7 @@ +# Changelog + +All notable changes to the **Apify for Cursor** plugin are documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [1.0.0] — Initial Cursor release diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..ce2a292 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2018 Apify Technologies s.r.o. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/README.md b/README.md index d4f8b14..3876184 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Apify for Cursor -Official Apify plugin for Cursor — adds the Apify MCP server, an `apify` routing subagent, and a set of skills covering the full Apify product surface: using Actors from the Apify Store, building and deploying your own Actors, and integrating Apify into existing applications. +Official Apify plugin for Cursor — adds the Apify MCP server, one `apify` routing agent, a routing rule that funnels Apify requests through that agent, and five bundled skills for the main Apify workflows: using existing Actors, building and deploying custom Actors, actorizing existing projects, generating Actor output schemas, and integrating Apify into existing applications. > **Apify** is a platform of thousands of serverless cloud programs called **Actors** for web scraping, browser automation, and data extraction. Learn more at [apify.com](https://apify.com). @@ -8,42 +8,49 @@ Official Apify plugin for Cursor — adds the Apify MCP server, an `apify` routi | Component | Path | Purpose | |---|---|---| -| Subagent (entry point) | `agents/apify.md` | Routes every Apify request to the right skill or MCP tool. **This is the one you should invoke.** | -| Routing rule | `rules/apify-routing.mdc` | Tells Cursor to hand Apify-related requests to the `apify` subagent first, instead of guessing between the per-task skills. | -| MCP server | `mcp.json` → `apify` (`https://mcp.apify.com`) | Lets the agent search the Apify Store, fetch Actor details, run Actors, and read the Apify docs. | -| Skill | `skills/apify-actor-development/` | Create, debug, and deploy a brand new Apify Actor from scratch. | -| Skill | `skills/apify-actorization/` | Convert an existing JS/TS, Python, or CLI project into an Apify Actor. | -| Skill | `skills/apify-generate-output-schema/` | Generate `dataset_schema.json` / `output_schema.json` / `key_value_store_schema.json` for an existing Actor. | -| Skill | `skills/apify-sdk-integration/` | Add Apify Actor execution to an existing application using the `apify-client` package. | -| Skill | `skills/apify-ultimate-scraper/` | Pick the right Actor from ~100 pre-built scrapers (Instagram, Facebook, TikTok, YouTube, LinkedIn, Google Maps, Reddit, Airbnb, Yelp, …) and run them end-to-end via the Apify CLI. | +| Agent (entry point) | `apify/agents/apify.md` | Routes every Apify request to the right skill or transport path. **This is the one you should invoke.** | +| Routing rule | `apify/rules/apify-routing.mdc` | Tells Cursor to hand Apify-related requests to the `apify` agent first, instead of guessing between the per-task skills. | +| MCP server | `apify/mcp.json` → `apify` (`https://mcp.apify.com`) | Lets the agent search the Apify Store, fetch Actor details, run Actors, and read the Apify docs. | +| Skill | `apify/skills/apify-actor-development/` | Create, debug, and deploy a brand new Apify Actor from scratch. | +| Skill | `apify/skills/apify-actorization/` | Convert an existing JS/TS, Python, or CLI project into an Apify Actor. | +| Skill | `apify/skills/apify-generate-output-schema/` | Generate `dataset_schema.json` / `output_schema.json` / `key_value_store_schema.json` for an existing Actor. | +| Skill | `apify/skills/apify-sdk-integration/` | Add Apify Actor execution to an existing application using the `apify-client` package. | +| Skill | `apify/skills/apify-ultimate-scraper/` | Pick the right Actor from ~100 pre-built scrapers across 15+ platforms and run them end-to-end via the Apify CLI. | +| Asset | `assets/logo.svg` | Plugin branding asset used by the Cursor package. | ## Installation ### From the Cursor Marketplace 1. Open the **Marketplace** panel in Cursor. -2. Search for **Apify** (the plugin id is `apify-cursor`). +2. Search for **Apify** (the plugin id is `apify`). 3. Click **Install**. 4. Reload the window when prompted. ### Manual / local install (for testing) ```bash -git clone https://github.com/apify/apify-cursor-plugin ~/.cursor/plugins/local/apify-cursor +git clone https://github.com/apify/apify-cursor-plugin ~/.cursor/plugins/local/apify # Then in Cursor: View → Command Palette → "Developer: Reload Window" ``` ## First-run setup -The plugin uses **two different authentication paths** depending on what you're doing. The `apify` subagent will guide you through whichever one is needed, but here is the high-level map. +The plugin uses **two different setup paths** depending on what you're doing. The `apify` agent will guide you through whichever one is needed, but here is the high-level map. ### Path 1 — Using existing Actors (MCP) Uses **OAuth**. The first time the agent calls a tool that needs auth (e.g. `run-actor`), Cursor opens `console.apify.com` in your browser and asks you to sign in. Read-only tools (`search-actors`, `fetch-actor-details`, `search-apify-docs`, `fetch-apify-docs`) work without auth. -### Path 2 — Building Actors, integrating the SDK, or running the ultimate-scraper +### Path 2 — Local Actor workflows and SDK integration -Uses the **Apify CLI** and/or an **`APIFY_TOKEN`** environment variable. Install the CLI once: +This path covers the bundled skills that do not rely on MCP: + +- `apify-actor-development`, `apify-actorization`, and most `apify-ultimate-scraper` workflows use the **Apify CLI**. +- `apify-sdk-integration` uses an **`APIFY_TOKEN`** with the `apify-client` package or the REST API. +- `apify-generate-output-schema` works against the Actor files in your workspace and may use `apify run` later only as a validation step. + +Install the CLI once if you're doing local Actor development or CLI-driven scraping: ```bash npm install -g apify-cli @@ -53,22 +60,23 @@ export APIFY_TOKEN="apify_api_xxxxxxxxxxxx" Generate a token at [console.apify.com/settings/integrations](https://console.apify.com/settings/integrations). Don't have an account? [Sign up free](https://console.apify.com/sign-up) — no credit card required. -This path is used by: +In practice: -- `apify-actor-development`, `apify-actorization`, `apify-generate-output-schema` — need the CLI for `apify init` / `apify run` / `apify push`. -- `apify-sdk-integration` — uses the `apify-client` npm package over HTTPS; only needs `APIFY_TOKEN`. -- `apify-ultimate-scraper` — drives `apify actors search`, `apify actors call`, and `apify datasets get-items`. +- `apify-actor-development` and `apify-actorization` rely on the CLI for commands such as `apify create`, `apify init`, `apify run`, and `apify push`. +- `apify-sdk-integration` only needs `APIFY_TOKEN` and the `apify-client` npm package. +- `apify-ultimate-scraper` drives CLI commands such as `apify actors search`, `apify actors call`, and `apify datasets get-items`. +- `apify-generate-output-schema` analyzes the local Actor codebase first, then can optionally be followed by local validation. ### Working in remote sessions, devcontainers, or SSH (no browser) The MCP OAuth flow needs a browser. If you're running Cursor over SSH, in a devcontainer, or in any environment where Cursor cannot open a browser, you have two options: 1. **Authenticate locally first.** Connect the Apify MCP server once on your laptop with a normal Cursor session so the OAuth refresh token is stored in your Cursor profile, then reconnect remotely. -2. **Skip MCP and use the CLI / SDK skills.** Every skill in this plugin can run without MCP — they use the Apify CLI or the `apify-client` package and only need `APIFY_TOKEN`. Set the token and tell the agent to "use the Apify CLI" for that session; the `apify` subagent will pick CLI transport automatically when MCP is unavailable. +2. **Skip MCP and use the non-MCP workflows.** The bundled skills can still work without MCP, but the transport depends on the task: CLI for Actor development, actorization, and scraper runs; `apify-client` plus `APIFY_TOKEN` for app integrations; and local Actor files for schema generation. Tell the agent to "use the Apify CLI" when you want the CLI path explicitly. ## How to use it -Almost every interaction should start with `@apify`. The subagent reads the request, decides which route applies, and dispatches to the right skill or MCP tool. +Almost every interaction should start with `@apify`. The agent reads the request, decides which route applies, and dispatches to the right skill or MCP tool. ``` @apify find me 5 well-rated coffee shops in Seattle and export to CSV @@ -79,13 +87,13 @@ Almost every interaction should start with `@apify`. The subagent reads the requ ### About the slash menu -Cursor currently lists every skill in the slash menu, so you may also see `/apify-actor-development`, `/apify-ultimate-scraper`, etc. Please **prefer `@apify`** — the routing rule will redirect you back through the subagent anyway, and the subagent owns critical guardrails (such as the `apify` vs `apify-client` package name trap that silently breaks projects when picked wrong). +Cursor currently lists every skill in the slash menu, so you may also see `/apify-actor-development`, `/apify-ultimate-scraper`, etc. Please **prefer `@apify`** — the routing rule will redirect you back through the agent anyway, and the agent owns critical guardrails (such as the `apify` vs `apify-client` package name trap that silently breaks projects when picked wrong). ## Components reference ### MCP server -The `apify` MCP server is configured in `mcp.json` (`https://mcp.apify.com`) and exposes: +The `apify` MCP server is configured in `apify/mcp.json` (`https://mcp.apify.com`) and exposes: - `search-actors` — search the Apify Store by keyword (no auth) - `fetch-actor-details` — Actor specs, input schema, pricing (no auth) @@ -97,25 +105,25 @@ You can disable or re-enable the server in **Cursor Settings → MCP**. ### Skill references -Some skills ship with extra reference docs that the agent loads on demand: +Some skills ship with extra reference docs and workflow playbooks that the agent loads on demand: -- `skills/apify-actor-development/references/` — `actor-json.md`, `input-schema.md`, `dataset-schema.md`, `key-value-store-schema.md`, `output-schema.md`, `actor-readme.md`, `logging.md`, `standby-mode.md`. -- `skills/apify-actorization/references/` — `js-ts-actorization.md`, `python-actorization.md`, `cli-actorization.md`, `schemas-and-output.md`. -- `skills/apify-ultimate-scraper/references/` — `actor-index.md`, `gotchas.md`, and a `workflows/` folder covering lead generation, competitive intel, influencer vetting, brand monitoring, review analysis, content & SEO, social analytics, trend research, recruitment, real estate, e-commerce price monitoring, contact enrichment, RAG, and company research. +- `apify/skills/apify-actor-development/references/` — `actor-json.md`, `input-schema.md`, `dataset-schema.md`, `key-value-store-schema.md`, `output-schema.md`, `actor-readme.md`, `logging.md`, `standby-mode.md`. +- `apify/skills/apify-actorization/references/` — `js-ts-actorization.md`, `python-actorization.md`, `cli-actorization.md`, `schemas-and-output.md`. +- `apify/skills/apify-ultimate-scraper/references/` — `actor-index.md`, `gotchas.md`, and a `workflows/` folder covering lead generation, competitive intel, influencer vetting, brand monitoring, review analysis, content & SEO, social analytics, trend research, recruitment, real estate, e-commerce price monitoring, contact enrichment, RAG, and company research. -The `apify-ultimate-scraper` skill drives the **Apify CLI** directly (`apify actors search`, `apify actors call`, `apify datasets get-items`) — no bundled scripts to install. Make sure `apify-cli` v1.4.0+ is on your `PATH`. +The `apify-ultimate-scraper` skill drives the **Apify CLI** directly (`apify actors search`, `apify actors call`, `apify datasets get-items`) — there are no bundled helper scripts to install. Make sure `apify-cli` v1.4.0+ is on your `PATH`. ## Troubleshooting **OAuth browser never opens / hangs.** See the "Working in remote sessions" section above. -**`apify: command not found` from the ultimate-scraper or actor-development skills.** Install the Apify CLI: `npm install -g apify-cli` (requires Node.js 20.6+). Then run `apify login` or export `APIFY_TOKEN`. +**`apify: command not found` from the actor-development, actorization, or ultimate-scraper skills.** Install the Apify CLI: `npm install -g apify-cli` (requires Node.js 20.6+). Then run `apify login` or export `APIFY_TOKEN`. **Authentication errors from CLI commands.** Run `apify login` to refresh OAuth, or set `APIFY_TOKEN` in your shell or project `.env`. Generate a token at [console.apify.com/settings/integrations](https://console.apify.com/settings/integrations). -**The wrong skill keeps getting picked.** That's exactly the problem the `apify` routing rule and subagent are designed to prevent — make sure you're starting with `@apify`, not `/apify-`. +**The wrong skill keeps getting picked.** That's exactly the problem the `apify` routing rule and agent are designed to prevent — make sure you're starting with `@apify`, not `/apify-`. -**`apify` vs `apify-client`** — these are two different npm packages. The `apify` package is the SDK for **building** Actors (used inside an Actor's code, on the Apify platform). The `apify-client` package is the API client for **calling** Actors from your own application. The subagent picks the right one for you; if you're installing manually, double-check. +**`apify` vs `apify-client`** — these are two different npm packages. The `apify` package is the SDK for **building** Actors (used inside an Actor's code, on the Apify platform). The `apify-client` package is the API client for **calling** Actors from your own application. The agent picks the right one for you; if you're installing manually, double-check. **MCP server appears disconnected after Cursor restart.** Open **Cursor Settings → MCP** and toggle the `apify` server off and on. If that doesn't help, re-trigger OAuth by running any Actor command. diff --git a/apify/.cursor-plugin/plugin.json b/apify/.cursor-plugin/plugin.json new file mode 100644 index 0000000..c92d46b --- /dev/null +++ b/apify/.cursor-plugin/plugin.json @@ -0,0 +1,20 @@ +{ + "name": "apify", + "description": "Official Apify agent skills for web scraping, data extraction, and automation", + "version": "1.0.0", + "repository": "https://github.com/apify/apify-cursor-plugin", + "author": { + "name": "Apify", + "email": "support@apify.com" + }, + "logo": "assets/logo.svg", + "homepage": "https://github.com/apify/apify-cursor-plugin", + "license": "Apache-2.0", + "keywords": [ + "apify", + "scraping", + "automation", + "leads", + "data-extraction" + ] +} diff --git a/apify/mcp.json b/apify/mcp.json new file mode 100644 index 0000000..e484889 --- /dev/null +++ b/apify/mcp.json @@ -0,0 +1,7 @@ +{ + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com" + } + } +} diff --git a/assets/logo.svg b/assets/logo.svg new file mode 100644 index 0000000..87eb4e4 --- /dev/null +++ b/assets/logo.svg @@ -0,0 +1,12 @@ + + + + + + + + + + + +