TestRail 사용 후기 및 리뷰

Regression testing

주석: TestRail was implemented to do the record taking for manual regression testing and planning.

장점:

Great product allowing us to keep history of our testing and specifically regression testing of our software development team

단점:

Find some reporting can be difficult, but in general it is a great tool for managing our regression testing test cases, and ensuring that they have all been run.

Testcase Management

주석: We enjoy using this tool because it takes little time to train new members and easy of using. SImple and faster.

장점:

Ease of use. Simple and intuitive flow for the workflows that can be customized based on your work.

단점:

Traceability is based on how you configure. Little confusing if you try to explain to users not familiar with the tool.

Used to be great, now it is horrible

주석: I've been told their team was decimated after it was bought out. They have only 20 people to handle 11,000 customers. A product that is supposed to be for QA, is not QA'd before releases. If you are considering this product my advice is to run in another direction. You will be disappointed if you invest in TestRail.

장점:

I loved it before it was sold. It was solid, easy to use, and made our work flow simple and efficient. That is no longer the case

단점:

Since it was bought out, it is now horrific. Test data disappears or is inaccessible. Test runs disappear. EVERY single release is extremely buggy. And knowing that, they still continue to push the release in waves to other customers instead of fixing high priority issues first. Tickets are opened and we get the canned "we're working on it" but rarely are bugs fixed correctly the first time (if at all). We have had months of work disappear with a single release. We've had things break where we were unable to access our tests for days or even weeks at a time. I went from convincing my entire company to join in to use this product, to now only having 3 of us left who will touch it. It's just sad how far it has fallen.

Everything we needed at half the cost

주석: We are better able to track our software testing and coverage with this tool.

장점:

I love the way they setup billing, every month you it automatically detects how many active users you have and you pay for that instead of being roped into a set amount of licenses. This has been incredibly helpful as we've gone through hiring, layoffs, and bringing on offshore contracted workers. Great flexibility! Another thing I like a lot are the UI Scripts, where you can customize the instance adding in or removing features and functionality. They are very powerful and super useful for us.

단점:

It bugs me that every time you close a test run the test case id numbers have a huge spike, a case we created today had the id number of over 11 million even though we have less than 50,000 cases total in our library. I also do not like the way you add and remove tests from a run. It is a bit awkward and hard to see exactly what tests you're adding and removing.

If you are executing manual test cases on a large scale then Test Rail is for you

주석: Having using various test management suites TestRail does excel in that it was built to run manual test cases and display results in an easy glance for the whole team. Linking to defects is simple, generating a report when the test run is complete. I feel like reusing regression suites could be improved, it does what it is meant to however you have to repeat several steps each time you want to generate a regression suite.
Some areas for improvement would be a test case type for API tests, I am sure everyone needs this but struggles to find a way to record this data.

장점:

I think report generation is the strong suit of Test rail, the percentage pie chart of passed, failed blocked cases at a glance make daily reporting for scrum stand up meetings ideal. Integration with Jira is excellent, although some room for improvement.

단점:

Trying to do something simple like import test case steps from a .csv is not so easy. You can import a suite but that is not ideal. Trying to duplicate cases can be tedious, let's say you have a lot of cases, trying to copy paste them in to various projects is a lot of hassle, it is surprising they have not addressed issues like this after so many years.

Best test management tool for QA

주석: It saves my time and its very useful for QA use this tool, actually you can create everything in this tool and not use other tools.

장점:

I'm using this tool everyday at my work, create test cases, tracking their status, and also create different reports to see the status. This tool works fast, has good user friendly design, by tracking results you can get an email notification.

단점:

Didn't find any drawbacks its really fast and convenient in using, also help to keep all info in one place.

Perfect QA platform for teams spread across the globe

장점:

- Web based - doesn't require a localized server installation (unless you want to) and can be accessed by anyone with credentials, no matter where they are located without any effort - Very easy to use and gives you a very neat category division when it comes to test cases, test runs and milestones

단점:

Can't search and replace words in multiple test cases at the same time which makes it harder to re-use existing test cases with only small changes

Easy to use and mantain. Very useful to do regressions.

주석: I use it every day. It's very easy to define the tests and execute them manually and with automations.

장점:

It's very easy to use, with different kind of configurarions and customizations.

단점:

The tests scenarios are not versioned. There's not possibility to see previous changes or have different versions of them.

Inexpensive, basic test management tool you should implement from day 1

주석: Saves us a ton of time/money in testing, helps our process stay consistent, and makes it faster to bring up new employees.

장점:

Easy and simple way to build a QA process, plus it's web-based, easy to admin, and has a simple UX that even our QA co-ops can follow for manual test. Has scaled well with our team.

단점:

Not much to not like. Works as advertised but doesn't seem to tie easily into automated testing, maybe that could be an add on in future.

Good for new eployees

주석: In my company TestRail work a bit slow (because of the VPN probably), so I do not enjoy really using it, but as a tool it is simple and useful. If I had a choice, I would be still using it with no doubt.

장점:

Although some functions should be clearly left for more experienced workers, new employees should have no problems with filling the test cases. They need a few minutes to get used to the UI, but in general it easy to learn.

단점:

Not being able to delete some of the elements I added (admin status was needed). Blocking this for elements added by other users would be justified, but if I make a mistake, I should be able to fix it myself.

Stellar Feature Rich Platform

장점:

The ease of use for maintaining, connecting and sharing test cases are unrivaled. The platform is so feature rich every time I think of a feature I want to request they already have it!

단점:

Takes a good amount of time to spin up a valuable framework. I think this can be mitigated by some test case best practices or maybe a super-user demo within the platform.

TestRail - test management better than competitors.

주석: I used TestRail in my previous company and I am using it in my current one. Firstly, it is easy to use, and all options and settings are close at hand. The structure of test management is also pretty clear. Projects -> Suites -> Sections -> Test cases. Easy to handle and maintain. The whole system is quite simple to integrate with e.g. Jenkins, Cypress, or Jira through the TestRail API. Some products are seamlessly integrated - like Jira, but for some products, scripts have to be written from scratch. Also, thanks to the fact that TR is widely used, there is plenty of help on the internet. Reporting and analysis can be adjusted to the user's needs and set to execute automatically, which can reduce the amount of work. Charts and analyses are easy to understand and can be expanded with the data you need. A big advantage is test parametrization and the ability to recognize and use test data sets. That allows having significantly fewer test cases to maintain. Product is being constantly developed and the customer service is good.
As for the downsides, sometimes minor bugs appear e.g. for some time there were duplicated input fields during test case creation/edition - nothing crucial, but it could be annoying. It was fixed two months later and now everything works correctly. The price is quite high and depends on the number of users.
All in all, in my opinion, TestRail is the best on the market providing user-friendly testing management.

장점:

The product is constantly developing. A lot of functionalities - API, test structure, reports, bug integrations, milestones, etc. Ease of use. Customizability. Integration with other software (e.g. Jira, Cypress, etc.)

단점:

Minor bugs. Quite a high price. Rather plain design (not really intuitive)

TestRail: An Effective QA Suite

주석: TestRail has been useful in my QA position for tracking the progress of test cases in our weekly regression runs. The framework it gives you for test case management is flexible enough to be used in a variety of ways for our company, whether for bug testing or tracking what tests have been automated over time.

장점:

- The system of test suites (a master list of all your test cases) and test runs (individual lists of specific test cases) is well-built and allows you to manage your test cases easily. - Product is consistently stable. - Test cases in different parts of the product have synced states, which makes it easy to make changes to a test case when necessary. - Tools for sorting test cases in a run/suite are powerful.

단점:

- There isn't a good tutorial for the product (or at the very least I never saw anything like a tutorial). As a result, it took me a while to notice some of the more unintuitive features. - Collaboration can be a bit clunky sometimes (multiple users editing a single test case, notifications at email level only).

Organize and Prioritize

주석: Coming from a Quality Assurance team which had zero test cases and no system TestRail has become a huge asset. It allows for our team which has members working in a different time zone to continually work independently while still having the entire team engaged.

장점:

Right off the bat the software is very easy to use The software allow us to manage our time effectively focussing on running regression tests on any particular task. It has been a useful tool for our Quality Assurance team to prioritize and organize test cases. The fact that we can implement the use of Jira is also a huge plus since this is a tool we also use within our organization.

단점:

Not many cons, the only two minor aspects that come to mind are one if you have no knowledge of how to use it does take some time to learn it. However if you have even the basic background on a tool like this the learning time does shorten. The other is the fact that results can only be shared with the registered users, so we are having to take screen shots of results to share with the rest of our company teams.

Very mature test management tool

주석: I'm very happy with it, it's stable, has a slick UI and overall it's a well-built product.

장점:

It can cover all of my needs in the entire test life cycle, it's very customizable - I've been adding a lot of my own custom fields to test case templates, for instance. It supports adding shared steps to make your tests more concise, lot of reporting options, milestones, test planning. Pretty much everything a tester needs.

단점:

I would like to see an option to add data parameters for my tests.

I have been using TestRail for more than a year. It is a good tool for test case management. Overall

주석: Good tool for test case Management

장점:

1. It is simple to login in. It takes your windows login/pwd 2. Test cases copying from the previous release is easy. 3 The pie chart display of TC run is precise and informative. 4. Its filtering and selection option is superb. And I like this feature very much.

단점:

1. I have administrative right to deleted folder, cases etc. But I feel delete option could have a bigger icon. Many times I have to search the tiny red button. 2. Artifact/screenshots addition option is good but after the run, you cannot add any artifact. You have to rerun test cases in testrail for future reference 3. Loading of GUIs takes little time [ but not often]

Must have tool

주석: We are using TestRail to manage automated tests on a variety of projects, from API tests to application acceptance tests. We've integrated TestRail into various CI platforms with ease.

장점:

TestRail is powerful yet easy to learn and use. It is extremely flexible and we have not encountered any application of it that was not a perfect fit. It has been widely adopted by our QA and Dev teams.

단점:

There are some admin features that could be easier such as managing users and grouping them for team access and policy rights.

Solid test case management tool

주석: Overall, we love the tool and have directed more teams to use it with the caveat that they do not have a lot of reporting needs.

장점:

Super easy to use and organize test cases. It works great and allows multiple structures depending on how your team manages test cases

단점:

The reporting capabilities are extremely limited. There are some canned reports that you can tweak. We struggle to find what we need without a custom report. You cannot download them as PDF. You need to download them as HTML which is not ideal in most work place scenarios. We can get most of the info we need from the dashboard, but it would be nice to download as a pdf to excel/csv.

Very user-friendly and powerful features

장점:

I love the integration with Jira and Ranorex. It makes the whole test execution process seamless.

단점:

Probably more of a process issue but I find that we can copy tests all over the place and start to lose our version control.

TestRail helped us get organized

장점:

Our employees enjoyed the user interface, and the ease to organize test cases and move them around. We have seen value in the ability to link up to Jira for tracking bugs that have been found and also linking to requirements that are also tracked in Jira.

단점:

The ability to have reporting at a higher level across all projects within TestRail. Need to have a better manager level view of everything.

Great test case management tool

주석: I used it in two companies already for 5 years now. In general i like the flexibility and that even non QA's can use the tool.

장점:

- easy to use - you can create a lot of testcases but you still keep the overview - you can connect it with jira - after some time the UI is easy to use and you can give the test case to tester

단점:

- design is outdated and could be a bit more intuitiv

Good management very robust tool that can match most sdlc

주석: Long time user and adapter

장점:

- clean interface - a few ways to equip/start projects - a lot of integrations - available API

단점:

- so as much as it is adaptable by doing so it also makes it not the strongest tool since it needs to support many different styles - reporting engine to do customizations is a learning curve

Excelent management tool

주석: We are using testrail to manage all of our core tests in the platforms and the experience is excelent.

장점:

Easy to configure and use projects, testsuites and testcases. Can manage all projects independent. Integrations are easy for automatize testruns executions .

단점:

I miss some integrations with others tools.

Test Rail review

장점:

It is easy to use and user friendly. Also I can create the report easily

단점:

When moving the test to another test suite will remove the test result history

Outdated software. Features don't work.

장점:

Nothing. I do not recommend buying this software.

단점:

We faced with several issues using TestRail and Testrail still doesn’t resolve them (And I’m not sure they will resolve them in future): 1. It is not possible to link test cases and other documentation to tickets in ClickUp 2. It is not possible to automatically generate reports for Autotests from Gitlab for Cypress every time the tests are run