Frequently Asked Questions

Yep! The easiest way to have Sweep write tests is by modifying the description parameter in your sweep.yaml. You can add something like: “In [your repository], the tests are written in [your format]. If you modify business logic, modify the tests as well using this format.” You can add anything you’d like to the description parameter, including formatting rules (like PEP8), code style, etc!

You should always review the PR. However, we also perform testing to make sure the PR works using your existing GitHub actions. To get the best performance, add GitHub actions that lint, test, and validate your code.

Yes! In the sweep.yaml, you can set the branch parameter to something besides your default branch, and Sweep will use that as a reference.

To retry an issue, prefix your issue reply with 'Sweep: '. This will trigger Sweep to retry the issue.

Yes! In the sweep.yaml, you can specify docs. Be sure to pick the prefix of the site, which will allow us to only fetch the docs you need. Check out the example here: https://github.com/sweepai/sweep/blob/main/sweep.yaml (opens in a new tab).

Yep! You have three options depending on the degree of the change:

1. You can comment on the issue, and Sweep will rewrite the entire pull request. This will use one of your GPT4 credits.
2. You can comment on the pull request (not a file) and Sweep can make substantial changes to the pull request. Sweep will search the codebase, and is able to modify and create files.
3. You can comment on the file directly, and Sweep will only modify that file. Use this for small single file changes.

We’re open source so that our users can see exactly how their data is processed, as well as learn from how Sweep works! We’re really excited about building a community of Sweep users(like you!).

Copilot lives in your IDE and writes small chunks of code at a time. This takes ~3-5 seconds, and you need to watch it the entire time. Sweep runs completely asynchronously, and handles the task end to end. This might take 10-15 minutes, but you’re able to walk away and come back to a finished pull request. Copilot also doesn’t have access to the latest documentation.

ChatGPT can’t write the actual PR, and you’d have to paste the generated code into your codebase and create a PR yourself. ChatGPT doesn’t have access to your codebase and the latest documentation, so it’s limited with large software projects.

AutoGPT (and similar tools) doesn’t work, and Sweep works. We don’t allow the language model to perform open domain tool execution (which doesn’t work well). We perform a fixed flow of search → plan → write code → validate code, repeating the last two steps. This lets us reliably generate PRs corresponding to the user description.

Sweep is built to improve on an existing codebase, which is a more frequent and higher need, than generating boilerplate, which is mostly a solved problem since you can just fork existing boilerplates.

We do more than just chat-with-your-code by actually creating pull requests.

We're shipping new features at Sweep everyday and unfortunately it looks like AutoPR hasn't received any updates in months.

We only support GitHub at the moment. We're currently focused on delivering more value and making Sweep more reliable and capable as opposed to supporting more services.

We currently only support GPT-4 but are actively testing on Claude v3 which is looking promising -- it's faster, less expensive and smarter with using the tool calls. Gemini also looks promising with long-context multi-file reasoning but we have tested on it less. We just moved off function calls for our code modification agent so this is now possible.

All our prompts are currently tuned for GPT-4 so exploring a new model involves re-tuning all the prompts and re-benchmarking each component -- from code search / reranking to planning to code editing. This is why reason you can't just plug LiteLLM. The majority of models out there, including even GPT-3.5, struggle to respond in the right format given or current prompts. Migrating the clients over is the trivial part.

This is also why we stopped supporting GPT-3.5: it's simply too unreliable and can't respond with our format consistently. It's not a matter of generating buggy or incorrect code but more of a matter of making any valid code edits at all.

We currently don't support or plan on supporting any open-source models. The only model that is viable is Deepseek but we have found that it's very hard to prompt-tune to use our diff format (or any viable diff format).

We're currently focused on building the most capable coding agent with the models available today, rather than building the most accessible or least expensive agent, or the one that runs on the lowest possible hardware requirements.

We access your GitHub repository at runtime. At the end of execution, your code is deleted from the server. To perform search over your codebase, we use the hashed contents along with the embeddings. This allows us to avoid storing any code as plaintext. Read more about it here: https://docs.sweep.dev/blogs/search-infra (opens in a new tab).

Yes! Check out https://docs.sweep.dev/about/examples (opens in a new tab).

We don't directly support it but we recommend setting up a Zapier plugin that syncs GitHub issues from Linear/Jira. This will automatically create the GitHub issues when you make a Linear/Jira issue which will trigger Sweep if prefixed with "Sweep: ".

The 'Empty repository' error occurs when the repository was just created and Sweep will not work. This is because Sweep needs some existing code in the repository to work with. If the repository is empty, Sweep does not have any code to analyze or modify. To fix this error, you can add some initial code to your repository before using Sweep. Once there is some code in the repository, Sweep should be able to work as expected.