We are building a responsive Ecommerce Website and would like to test it on several mobile devices on cloud. There are so many of them like AWSDEvice Farm, Sauce Labs, CrossBrowserWeb Testing etc.
We need the tool to perform some manual tests on real devices as well as run our Automated Selenium Scripts on real device.
Any thoughts or advice which ones are better from the ones mentioned above
Thanks
QA
That's really hard to answer as it comes down a matter of opinon and all the specs of the project.
I don't know about the other but device farm has you upload the test package which can only be a appium java testng/junit or python pytest package. Then in the public offering it will find a device and run the tests which may take longer than other options because it restarts the appium server between tests.
In the private offering for device farm, we can use direct device access to run any testing framework we want. However, the up front cost for these is more than the public offering.
You are also limited in the execution time. Right now we can only run the tests up to 150 minutes.
Hope that helps
James
Related
Greeting Community,
My boss is asking me to do manual checking/testing of our intra-net site
The company is very low budgeted and don't want to spend any money.
What would be open-source free regress test tool it can run stand-alone on a window system? The company uses internet explorer plan to standardize on Window.
Our server system is very limited and we have difficult time to allocate disk space even for 100G.
We support 1000+ users.
Would Selenium be a good choice for this task ??
I am new to regress test on webpage so something simple to setup and learn will be very helpful
Thanks.
Yes you can a combination of those free and open source tools that can run on Windows with Node.js:
selenium-webdriver,
mocha,
chai.js with chai-webdriver.
That's all you need to perform full end-to-end tests. I use this with Internet Explorer, but it works with Chrome and Firefox, too.
I'm new in tests area. Regression team where I belong has built GUI tests for some web applications with complex business logic that developers team has produced.
Until now, we have been using Selenium IDE to build regression tests (record, edit, parameterize, debug and playback). Tests are exported and maintained in Html format. We used to have a tool to manage tests and iterations (store html scripts/tests suites, run tests in batch mode, run tests in background, get detailed test result reports), which is now deprecated because uses Selenium RC. Additionally, tests are made only in Firefox, but our clients are mainly IE users.
So, we have some important and strategic decisions to make. We need urgently to start testing in IE and a new way to do the tasks we were doing.
An attempt was made to change the code of tests’ manager tool in order to work with Selenium Webdriver. It was tried to code tests in Ruby from the beginning, since Selenium IDE export to Ruby was not satisfactory. We figured out that huge changes and subsequent tests on the manager tool were needed. It would also involve programming the methods and test them.
Our regression team is quite small and we don’t want to focus too much on the programming task itself, but more on testing our webApps. Additionally, no one on the general team had experience in working with Ruby before.
Can you help us with some suggestions about the route we should take?
Is there an integrated solution easy to work with (as Selenium IDE) and able to do the manager tasks of our old tool without taking us much time on “hard coding”?
Is there any reliable open source tool that could do it? And a commercial solution?
I am using selenium to automate my application.. My applicaton works on IE only.. I dont have much scope for other browser..
but when my scripts are running if i do other activities like reading mails, or update QC in another window.. Those are breaking my scripts (no such element or no such window..) once in a while (mostly out of 10, 2 to 3 times i am facing this issue). But can not make system idle till my scripts are run as i do have other activities after started the script. How to stabilize this?? Any one facing these kind of issues??
Running Selenium tests locally using your one and only IE browser does come at the cost of having to not touch your mouse or keyboard during test executions. A way to get around this is to create one or more virtual machines. VirtualBox from Oracle is a popular choice but there are others. You need to install an operating system on your new virtual machine and odds are that your existing Windows license is single use. You can request an additional Microsoft Windows OS license from your IT department or simply buy one yourself for, what? $190? I have done that when the paper work at my client was unmanageable. My time is worth more than that.
Another alternative is to take advantage of 90 day free licenses from Microsoft.
Start your test on the virtual machine then change focus back to your desktop to do other work. I have even added code to the end of test logic to beep when the test completes so that I know when to expand my virtual machine.
Update
You should add an antivirus to that virtual machine. Safety first. :-)
Selenium does not support and non web-based applications, it only supports web based applications.
So if you are doing activities like reading mails and update QC in other window, you will not able to find the elements through selenium.
If you are facing some challenges with IE browser. Please refer this link:https://code.google.com/p/selenium/wiki/InternetExplorerDriver
There are following limitations are given below:
Some limitations of Selenium Automation tool are as follows:
It does not support and non web-based applications, it only supports web based applications.
Its and open source tool so in case of any technical issues you need to rely on the selenium community forums to get your issue resolved.
You need to know at least one of the supported language very well in order to automate your application successfully.
No inbuilt reporting capability so you need plugins like JUnit and TestNG for test reports.
Lot of challenges with IE browser.
We currently run and deploy on app engine, but use GitHub as version control. What is the best way to run a series of tests every time we push to GitHub, both client-side Javascript tests, using something like PhantomJS as well as something like NoseTests for Python?
The reason being that client side code is in Javascript while the server side code is in Python.
And since we have existing credits, we'd prefer not to go for a 3rd part hosted solution. App Engine also provides a pipeline for just node tests, but this doesn't cover the Javascript unit tests.
Thanks!
I believe github commit webhooks are what you are looking for. I have not personally set them up, but at my day job, we have it automatically run a handful of things including builds + tests.
https://help.github.com/articles/about-webhooks/
There is a Google Scrip to accurately test the backend loads. Unfortunately I don't know of anything from JS.
In the documentation for App Engine and in presentations we've given at Google I/O we have mentioned that you should ramp up slowly when load testing an application on App Engine. Ramping up too quickly won't give an accurate picture of how App Engine scales; you have to accomodate our load balancing code which determines how many instances of your application to spin up by watching how much traffic is directed to your application. That monitoring and adjustment takes time, therefore the need for not ramping up too quickly.
I've looked at various load testing tools and in the end wrote my own short script in Python which I use as a base for all my load testing. This isn't to say that what I have here is better for load testing than the available packages, please look at them and judge them against your own criteria. I'm most comfortable with Python and a skeleton script that can be tweaked for each test scenario is optimal for me.
We are developing some testing infrastructure and I have hit a coders block (lack of sleep?)...this seems like it would be a solved problem but I haven't found what I'm looking for via google.
I would like to automatically push builds from our CI server (TeamCity) to a number of machines (growing, but currently 30). These are several WinForms apps and a number of dlls. Once deployed, I would like to kick off tests (NUnit, for both unit and integration tests) and report all results (back to CI? or somewhere else? Not sure).
The target machines are a number of platforms (Win7,Vista, XP, Server 2k8, Server 2k3, Ubuntu, Fedora, Suse, x64, x86, maybe macs down the line)
This gets me part way there (the actual push). But I can't find existing solutions for 'push starting' the tests and reporting back. So far I am thinking of combining the link (or similar) with custom code running on each client machine that watches the deploy directory, runs the tests and reports the results.
Does anyone know of existing solutions?
Links?
Done something similar and care to share?
Edit
If possible, we prefer .net based solutions, but it isn't strictly necessary. I would have tagged the question as such, but ran out of tags :)
You could use KwateeSDCM to both push and start on all the platforms you mention, including mac. However, you'll have to do some coding to get reports out. I'm not familiar with TeamCity but maybe you could push a script along with your application which could then transfer the test results via ftp to a server accessible by TeamCity.
Have a look at: STAF (Software test Automation Framework)
The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring).
Which includes STAX:
STAX is an execution engine which can help you thoroughly automate the distribution, execution, and results analysis of your testcases.
And there's an article here:
http://agiletesting.blogspot.com/2004/12/stafstax-tutorial.html
Assuming you have the push part done already, and you don't mind using a TeamCity license, you can create a TeamCity Command Line Runner build configuration or NUnit test configuration that kicks off the tests on a properly configured agent. The build trigger for this test config would be successful completion of the application build.
So far I have ended up using a seperate build step in TeamCity that executes a bat script that in turn fires of tasks to the list of machines using PsExec. So far my trial runs it is working ok, though I now need to parallelize the copying of build output...
Thanks for the input to those who have provided it.