I have been using selenium webdriver to automate AUT. Unfortunately our system has lot of known bugs and Product is ok with them but the real problem is those test cases will fail in TestNG report and management will get a big list of failed testcases. So Management asked me to remove known issues from the test but my test manager is not agreed to remove/disable them from test.
So what I am thinking is, create a custom annotation and have a field called bugNumber and based on the existence of the bug number I will create a report where test cases failed with known bug number will go down in the report and failed test cases without bugNumber will come on the top of the report. But I really don't know how to get those custom annotations to TestNG test-results.
Could you guys suggest me something please ?
Related
We recently refreshed our SF development org and I am trying to upload some new work (unrelated to Agile Accelerator managed package) to the org. I am running all tests on validation and getting a number of error messages on a number of ADM classes from the managed package for Agile Accelerator.
Here are a few of the error messages:
When looking at the object I can't find any field called ProfileId, or any reference to a ProfileId for a User. Obviously since the package is managed I can't get inside the code to determine where the issue lies.
Any help? Any idea of where to start?
You haven't pasted any errors... When you validate, which option you select?
Running all local tests should be enough. don't bother running all tests including managed package ones, it's almost guaranteed they'll fail (when package inserts new account it'll have no idea about your custom fields, your required, your validation rules...)
If you have run local tests and got errors - well, maybe there are few fields package added that now you have to populate to successfully insert? Generally see what successfully saves in UI and try to replicate that in test?
I wanted to try out integration of Selenium Jmeter and StormRunner. My end goal is to do Load testing with 'n' number of users on StormRunner
What ? - For e.g. I have Selenium Script, convert it in to Jmeter (I can get this information from many sources)
Then my Jmeter script should get ready
Then upload Jmeter script in to StormRunner and pass the necessary parameter through Jenkins and run the load test.
I really want the opinion here about feasibility and whether it is in right direction or not.
Idea here is that Automated Load/Performance test
Selenium is a browser automation framework and JMeter acts on HTTP protocol level so your "Automated" requirement might not be fulfilled especially if your tests are relying on client-side checks like sorting or waiting for element to appear.
Theoretically given you properly configure JMeter it can behave like a real browser, but it still not be executing client-side JavaScript.
If you're fine with this constraint - your approach is valid, if not and the "automated functional test" requirement is the must - consider migrating to TruClient Protocol instead
Why wouldn't you covert your script to a native Loadrunner/Stormrunner form of virtual user?
You should look at the value of what you are trying to achieve. The end value of a performance test is in analysis. Analysis simply takes the timing records and the resource measurements produced during the test, bringing them together on a common timestamp, and then allowing you to analyze what resource "X" is being impinged when timing record "Y" is too long. This then points to some configuration or code which locks up on resource, "X."
What is your path to value in your model? You speak about converting a functional test script to a performance one. Realistically, you should already know that your code, "works for one," before you get to asking, "Does it work for many?" There is a change in script definitions which typically accompanies this understanding.
Where are your collection of resources noted? Which Resources? On which Hosts? This is on the "path to value" problem where you need to have the resource measurements to diagnose root cause of poor performance.
I am using Selenium framework for my test cases execution.
I need an instant report of test cases that are passed while the full suite is in execution currently.
For Eg: There are 100 test cases in suite and five have run of which 3 passed, 2 failed and I need these instant report while the suite is in-progress. Can you please help me with this task?
You can use ExtentReport.
You can use it to log your test steps and once its done it will generate a report to show your results.
For what your looking for, ExtentReport uses a "flush".
If you call this flush after each test step it will amend the step and create the report.
This is something I'm looking into myself at the moment, so I wouldn't consider this an answer but something I've stumbled across myself, hope it helps.
Here is how to set up ExtentReports on your project with examples - http://www.ontestautomation.com/creating-html-reports-for-your-selenium-tests-using-extentreports/
You must use it in conjunction with a test runner eg. TestNG or JUnit.
For what you are trying to achieve is slightly different to the example. You need to call a flush after every test step so it will amend to the report after the step is completed rather than when all the tests are completed. Its not something I have done before but it was explained to me like the following
Just call .flush() after every test instead of once at the end of your test run. BUT you need to make sure the ExtentReports object itself is only initialized once, instead of being reinitialized at the start of every test. For example, I used TestNG. The ExtentReports is called once using #BeforeSuite, but the .flush() is called after every test using #AfterMethod. I hope this makes sense.
The only thing that can’t be solved via code is the HTML refresh as this is outside the control of the ExtentReports library (it doesn’t know where you’ve opened the actual HTML file). But this can be taken care of by using a simple browser plugin as I said. At least for Chrome there are a lot of them, just do a Google search for ‘chrome auto refresh’.
Hope this helps. If you need anymore advice don't hesitate to contact me.
I am currently working on writing automated tests using Selenium Webdriver. We use MTM to run our test suites. I need some ideas as to what would be a good way to write these tests.
Currently before running these tests, we perform a basic setup that sets the username and password that would be required to login to the site, set the browser that the test should use, and few other things.
Currently the data that is required for each of the test is setup manually and is already present in the database . The test simply performs a keyword search, finds the necessary data it needs and then performs the assertions. What we would like to achieve is find such data that is already present in the database and use it instead of creating it manually. That way I can run these tests across different environments(dev,qa,production).
The site I am testing is an e-commerce website. I mostly write tests for specific features that my team develops, and thus many of these tests require some specific data. e.g setting up a store that has products with certain shipping rates, with particular offers etc. I would like to find a way to automate or almost remove this manual process of setting up the data. That way I have the flexibility to run these tests across environments. Could you please direct me to some articles/suggestions that can help me achieve this ?
If I am understanding your question correctly, you want to automate the test data setup.
You can achieve this in following ways:
If possible, write a sql script which inserts the desired data in db. Now you can execute this while running your tests. If you are using TestNG framework, then there is already an annotation available like #BeforeTest. You can execute that sql script in this annotation, it will be executed once before your test and data is ready.
Prepare data in a spreadsheet. Create an algorithm, fill the data dynamically in spreadsheet and from there either read directly and fetch it to your test using #BeforeTest or if required, data in spreadsheet can be inserted in db also.
I have a database web application and I need to see all the possible inputs and all the possible outputs of this application (using Selenium or Jmeter).
Actually I tried to understand how the "Input Coverage Method" works in software testing tools but it seems too tough. If I'm not wrong this kind of testing I'm trying to do is a kind of Data Driven testing (means figuring out all the possible input and output of an database web application).
Would you please give a suggestion if there is any tool (I prefer open source) that can do this or any method to create such that test?
Do I have to create it by my own?
First of all you need to create Equivalence Classes that cover most of your input dataset.
After that you can simply run your selenium/JMeter tests with the test data created.
You just need to create single test script and populate the test data in excel or CSV sheets to perform data driven driven testing.
Have a look a jBehave.
It's a BBD tool that can drive selenium and supports sets of input test data.
I've used it and it works well. You'll need patience to get through the glue code, but once you're out the other side you'll be glad you persevered.