Frequently Asked Questions
Redefine is the first of its kind AI-based test selection platform, built to speed CI time and dramatically reduce cloud costs. Redefine’s algorithm identifies and runs tests that matter for each specific code change, offering real-time pass/fail input as each test is completed. To do this, Redefine studies your historic code changes and test suite to form a test selection model that helps your team predict failure sooner. This speeds up the inner development loop, improves code quality, and helps your team ship changes faster.In less than 15 minutes, Redefine integrates with your existing CI environment and testing framework. Our AI analyzes your code change history and test results to understand your team’s coding patterns, find dependencies and correlations, and recognize common failures.Every time a new code change is pushed, the test selection model automatically selects only the most relevant tests from your test suite. These tests will rapidly determine whether your code build will fail or succeed. This way developers can move fast making changes during their test run and avoid breaking things.
Redefine basically studies your historic code changes and test suite to form a unique test selection model that helps your team predict failure sooner.
Redefine tests only what is relevant for every code-change / run, so you can enjoy a much faster dev cycle and during your CI. The AI-based test selection model automatically selects only the most relevant tests from your test suite. These tests will rapidly determine whether your code build will fail or succeed. Developers get real-time fail responses as each test completes, so they can make changes during their test run.
Redefine learns new tests, with exploration mode, by running new tests a few times to collect information about their dependencies.
Redefine automatically detects flaky tests and provides a configuration to include/exclude them from redefine builds.
Redefine time budget (measured in seconds) is selected by the user and determines how long the longest test suite may take. Redefine uses this parameter to make sure build times are consistent across different code changes, while only running relevant tests.
Redefine coverage is the number of the CI builds where redefine selected all of the failed tests and surfaced all of the bugs out of all of the CI builds.
Note that increasing the time budget will always increase the coverage, meanwhile, lowering the budget will inherently decrease the coverage.
As many relevant tests are selected for the diff the accuracy metric is defined by Redefine Coverage.
The coverage can be adjusted according to your desired build time.
Recent result was 98% coverage with a 3 minutes time budget (out of 11 minutes average build time).
It depends on your desired time budget, so in case you have 100 tests, and we used about 30% of total build time it would be around 30 tests.
Yes, you do :). Having a Continuous Integration (CI) in place is inevitable, it helps ensure that the codebase is always in a deployable state. It also helps to identify any errors and problems in the codebase quickly, so that they can be addressed before they become major issues. Additionally, CI allows developers to focus on their own tasks, as the integration process is automated. This helps them to produce better quality code and deliver it faster, which is beneficial for the entire project.
Yes, Redefine supports all CI platforms.
We currently support Pytest and Selenium. Mocha, Jest and Cypress are coming soon.
You don't need a dedicated engineer to operate Redefine. Redefine is an AI-based tool and as such it is designed to self discover and learn the data it needs in order to operate and automatically train itself for self-improvement (without any human touch or maintenance).
It takes 5-6 minutes to install Redefine. The first step is to add the Redefine CLI tool to your existing CI, which takes about 3 minutes. After installation, Redefine enters its discovery stage, collecting all build histories and coverage (this stage runs in the background and you are not required to do anything during it). The final step is to activate the Redefine tool on your CI production environment and start saving cloud costs and accelerating development.
Redefine is SOC2 compliant, as we have a strong background in cybersecurity and understand the requirement to ensure customer security, reliability, and compliance. Generally, SOC2 standards are designed to ensure that tools are secure and in compliance with industry-specific regulations.
Redefine platform is hosted on AWS. There is a great value in having your CI optimization operated on the cloud: the first one is cost savings: Redefine, like other cloud-based tools, is more cost-effective than on-premise tools because it requires no upfront capital expenses and is billed on a pay-as-you-go basis. The second benefit is scalability: Redefine is easily scalable, allowing businesses to quickly add or remove users and resources as needed without any additional hardware or software investments. The third and last is flexibility: Redefine is highly flexible, allowing businesses to access their data and applications at all times without any limitations.
In order to increase velocity while protecting production, we recommend running redefine on every push to the feature branch, while running the full suite on merge to the main branch. If the organization is running scheduled builds (i.e. nightly), we recommend running the full suite there.
At Redefine, we take security very seriously; redefine is certified COC2 and the founders' combined vulnerability research experience exceeds seven years.
Yes, Redefine offers a free plan. A free plan is available to developers with fewer than 50 builds per month. You can use Redefine for free if you meet our application requirements. Our Professional plan for larger organizations offers a free trial of one month.
Most of our customers experienced saved dev time from the first code change run they shipped with Redefine, the best result so far was 91% saved time equivalent to 4 years, 345 days, 18 hrs and 13 min.
Most of our customers started experiencing 90% saved time once the discovery stage ended and the dev-teams are running their CI with the Redefine tool.
We recommend running the full suite as a last-stop before merge, or as a scheduled (i.e. nightly) build if one is used.
The pricing model is based on the number of CI runs per month, we offer 2 packages: one for professionals: priced per build and the second package is for enterprises, big engineering organizations who are looking for an enhanced solution to suit their needs.
