We are currently running smoke tests using Selenium Webdriver & JUnit against a B2C product. Since we are using Selenium, the scripts are totally dependent on the UI. Given that the product is out of a tech startup, the UI & workflows keep changing/evolving # an extremely high frequency.
The Consequence: Smoke tests which are supposed to validate the sanctity of the application keeps failing. The team spends more time fixing the scripts rather than validating the build.
I am pretty sure most of the Automation folks out there would have faced similar issues esp. with rapid dev cycles. Looking forward to see some approaches undertaken by others in the industry who have faced similar problems.
Note: The frontend is developed in PHP
Webdriver works roughly like this: there is a start point, webdriver interacts with it (by simulating a button press for example) and then finds the next item to interact with. The next item might be on the next page or the same page. It might be found in various ways, by id or the 3rd div that is class="foo" etc.
The tests are things like does the page load with 200 OK, does the string "login" appear in a particular place and so on
The problem with a changing UI is that all the elements "move about". The ids change and the 3rd div class foo disappears. This means that the webdriver interactions fail and the tests if they are looking for particular elements will fail too
One solution is to develop and test against a set of ids. These ids will refer to fixed UI elements. All searching in webdriver should use the ids. The development team writing the PHP will put the ids in the correct places.
The set of ids can also be used as the basis for a sort of specification and can be used to explain UI flow in different ways to different stake holders.
I do not know of any specific product that handles this process of managing ids in both tests and development code but maintaining a "lexicon" like this to describe the UI items should not be a major task
The more versatile the System under Test is the more important it is to have a framework on top of Selenium that reduces the maintenance effort for a change.
For the most common changes in a System under Test there are several known patterns that can help you to reduce the maintenance efforts:
By using UIMaps to model the UI of the application it is extremely easy to handle changed IDs, CSS classes or similar changes
PageObjects reduce the effort for larger UI changes (e.g. when an input field is changed from a TextBox to a Dropdown field)
Use Keyword Driven Testing to model test cases without any knowledge of the underlying technological representation. i.e. a keyword encapsulates an action from the users point of view – a example for a keyword can be: “loginWithValidUser()”
Don’t just utilize the UI for smoke testing if the UI / Application / Workflows change drastically and very often. Most of the time it is also helpful to test certain functionalities by calling WebServices without any Web-UI
Related
I have set up a GitHub action that validates code changes upon a pull request. I am using the Salesforce CLI to validate (on PR) or deploy (on main merge).
The documentation gives me several options to determine testing for this deploy.These options are NoTestRun, RunSpecifiedTests, RunLocalTests, RunAllTestsInOrg. I am currently using RunLocalTests as so:
sfdx force:source:deploy -x output/package/package.xml --testlevel=RunLocalTests --checkonly
We work with some big orgs whose full tests take quite a while to complete. I would like to only RunSpecifiedTests for validation but am not sure how to set up my GitHub action to dynamically know which tests to pull in. I haven't seen anything in the CLI docs to determine this.
There really isn't a way to do this with 100% reliability. Any change in Apex code has the potential to impact any other Apex code. A wide variety of declarative metadata changes, including for example Validation Rules, Lookup Field Filters, Processes and Flows, Workflow Rules, and schema changes, can impact the execution of Apex code.
If you want to reduce your test and deployment runtime, some key strategies are:
Ensure your tests can run in parallel, which is typically orders of magnitude faster.
Remove any tests that are not providing meaningful validation of your application.
Modularize your application into packages, which can be meaningfully tested in isolation. Then, use integration tests (whether written in Apex or in some other tooling, such as Robot Framework) to validate the interaction between new package versions.
It's only this last that can provide you real ability to establish a boundary around specific code behavior and test it in isolation, although you'll always still need integration tests as well.
At best, you can establish some sort of naming convention that maps between Apex classes and related test classes, but per the point above, using such a strategy to limit test runs has a very real possibility of causing you to miss bugs (i.e., false positives).
What sort of challenge will occur while automating Single Page Application tests using WebDriver
Hi answer to your question can very person to person but i think you should take care of following things :
1.for personal front please do not use POM page object model framework for SPA use data driven or even keyword driven both will work fine.
2.use small re-usable methods so that u can skip code redundancy.Also it will help in overcoming various exception like stale element exception.
3.also try to divide your SPA in various blocks (for example header ,footer,grid etc -hence when you will do any operation you know where you have to go it will give you a clear idea and easy code maintenance)
for now i can think of these points hope this helps you
My team was considering using angular js for web app UI development. But knowing at a high level how single page apps work, I had a question as to, how feasible it is to use angular js framework, for a large web application. this would have at least 200 screens, each screen containing an average of 30 UI controls like text boxes, calendar controls, drop downs etc.
The user will be accessing the web app on desktop or laptop and will be using the application in the browser throughout an 8 hour day, without ever closing the browser.
Given above usage, would angular js, memory usage / performance be issue?
are there web apps with huge and complex UI, built using angular js, that are in production and used everyday?
You can have not just 200 but 1000's of screens - this number does not matter as long as you address the core and fundamental questions below. They all boil down to memory and performance.
At a given point of time how many $watches will be active.
At a given point of time how many listeners are created.
At a given point of time what is the complexity of DOM i.e. the number of DOM elements and thee nesting/depth.
At a given point of time how many Javascript modules (services, controllers etc.) will be loaded in the memory. Even though these things are singletons they consume memory.
For each such screen how much memory will be consumed (if you use jqueryUI etc. your memory increases quite rapidly than pure angular + html components)
This brings to what you can do to control the above factors which if not watched will make your application sink/crash.
1) How do you break the run-time complexities of your "big" application ? You can now think of "Routers" or dialogs. Each of your screen will come-and-go i.e. you will never display 200 screens the same time.
2) Make sure when the screen is gone there is no "leftover". Don't use jQuery - if you do make sure you clean this on $scope.$destroy.
3) Multitudes of directives:- Directives are powerful but don't over use it - try not to deep nest too many of them. templateUrl is "tempting" but sometimes in-lining a template is the best choice. Use build tools that will inline the templates.
4) Load modules on demand using requireJS. This will force you to modularize your application and think hard about concatention strategy (combining JS files). Will force you to write independent modules.
5) Browser caching your assets and a good cache busting mechanism. Grunt based plugins are to your rescue. Minify your assets.
6) Compresss the assets for efficient network utilization and faster transmission.
7) Keep your app encapsulated in Angular. Don't create any global objects. Chances are that they have access to some closure or are referring to some things within angular $scope and $scopes are now still hanging on or are in difficult trajectory to be able to get Garbage Collected.
8) Use one-time-bind or bind-once model binding as much as possible.
9) Splash screen is an excellent weapon - us it. Helps you load some stuff upfront while the user watches smooth/jazzy picture/picture :)
Regarding 8 hours a day constant use
Unless there is a leak of the following kind you should be fine:-
1) Listeners leaking i.e. hanging around.
2) DOM leaks. Detached DOM issue.
3) Size of Javascript objects. Javascript objects coded in a certain way creates repeated instances of function.
(I am developing AngularJS app - with more than 450 screen - MDI like app. The app is not yet in production. The above points are coming from my burnouts in the functionality we have developed so far.)
I've worked on multiple projects that are extremely large single-page applications in Angular and Ember.JS at companies like McKesson an Netflix.
I can tell you for a fact that it's completely "feasible" to build "huge, complex" applications with frameworks such as Angular. In fact, I believe Angular is uniquely well suited to this because of it's modular structure.
One thing to consider when creating your Angular application is whether or not every user will benefit from all "200 pages" of your application. That is to say, is it necessary to have all 200 views be part of the same single page application? Or should you break it into a few single page applications with views that share concerns.
A few tips:
Watch out for name collisions in the IOC container: If you create enough services and controllers, even if you break them into separate modules, it's very easy to create two services with the same name. This may or may not result in an outright error. What happens when you register two "fooService" services? The last one wins.
If you decide to break your app into more than one single page app, you have to be sure you have solid boundaries of functionality between the two. They're not going to be able to share state easily other than with something like cookies or local storage.
If you decide to keep everything in one single page app, you're going to want to keenly watch your initial download time. Always check to see how long it takes to start your app "cold" over a slow-ish connection. If the entire app is in one bundle, depending on how you structure things (are you going to use AMD?) then you might see a long initial load time.
AVOID rendering HTML on your server. I can't stress this enough. Let Angular do that work for you. The only rendering your server should be doing is rendering JSON to be returned from some RESTful service.
Flush out your JS build process early on. Look into Node-based tools like Grunt, Gulp, and Broccoli for linting/concatenating/minifying your JS files. Checkout Karma for running unit tests, and look into Istanbul for code coverage. Protractor is a great tool as well, but I recommend strong unit tests in Karma over integration tests with Protractor just because Web Driver based tests tend to be brittle.
Other than that, I think you'll find a single page app written in any modern framework to be extremely snappy after the initial load is done. In fact, it will make any "old" PHP/ASP.Net style app that renders the entire page at the server seem slow as dirt in comparison. This is because you'll only be loading data and the occasional .html file over HTTP.
I'm a SP developer and also skilled in webdesign. My current task is to implement a responsive design along with a branding in a SP 2013 environment
Now I am in the situation to choose which one i would use to implement a responsive design in SP2013 and so I collected pro's and con's foreach technology.
What do you think ? Are there important aspects I'm missing ? Are there other solutions which are better suited for realizing this ? Any input is welcome ! :)
Device Channels
Yes I'm talking about Device Channels even when they are not mentioned in the question, because they can deliver the best performance and optimizability for the enduser and the client - in my opinion :)
Pro
individual designed HTML/CSS and JS foreach device
-- no need for hiding or removing incompatible elements
-- faster because you just load things you really need
-- faster because you will likely have less CSS/JS and HTML
-- faster because you can use optimized code foreach device
-- better you can better point out which channel has errors and changes dont affect the other channels
Con
individual designed HTML/CSS and JS foreach device
-- you have to append changes to each masterpage
-- more work to accomplish the same result (in general)
-- redundancy
bound to User Agent Strings
growing diversity of devices
-- may equals growing diversity of masterpages >> work
These are not all but my main points. Klick here to start your own research.
Bootstrap
Pro
mighty, easy to use framework
-- a lot of documentation
-- fast results
-- if you like it - all the Bootstrap styles
there are already projects using it so you may dont have to build it from scratch
-- http://responsivesharepoint.codeplex.com/
Con
Bootstrap is a huge framework and has 8000+ lines of code in the unminified CSS and JS files
-- 2 requests extra for ~ 130kb & 30kb
-- a lot of styles and script for your browser to handle
Bootstrap is not build for use in SharePoint
-- it's overwriting SharePoint styles which makes some features (ComposedLooks for ex.) less valuable
-- there are a lot of custom CSS needed to make it work seamless with SharePoint
SharePoint has it's own weird way to do things and that interferes with BootStrap
-- tons of CSS
-- tons of JS
-- tons of HTML Attributes
These are not all but my main points. Klick here to start your own research.
Media Queries
Pro
only necessary CSS
no JS if you dont wan't to
you can create your own layout
with response.js even in IE6 working
you can easily separate which features should be available in certain screen sizes
Con
several sets of CSS depending on the number of Breakpoints
every feature needs to be developed by yourself
it's not easy to write generic code that can process every SP2013 Page
-- it depends on the complexity of the content shown. I write about 150 lines of CSS that created a mobile view for publishing pages that contained the navigation and content, but no features like editing, etc.
-- if the client's want every feature on his smartphone, there is a hell lot of work and testing needed. (Plus who the hell wants to do that on their phone?)
Conclusion
I'm not sure yet (and it would be awesome to get a lot of feedback to my results), but i tend to use Media Queries. Why ? Well SharePoint has it's own way to handle desktop users and i wouldn't customize that build in functionality if not explicit ordered. On the other hand SharePoint doesn't provide a real UI for smartphones. I don't want to use BootStrap because it contains a lot of styling which will produce problems in branded environments. And I won't use Device Channels because of the downsides.
Use Botostrap. I work at a University (http://www.cmich.edu) and our entire website (internal and external) is built in SharePoint 2013 using this Framework, and I was the lead on its implementation. If you haven't used Bootstrap at all, it takes a little getting used to, especially in the SharePoint environment, but as far as I'm concerned, responsive design is the only way to go that's going to produce the best results. Check our University site out, if you have questions, let me know.
I have a (closed source) WPF application containing mainly two modules: A UI exe and a "Model" dll. One screen (for now) and about 30 classes.
I would like to start testing it with testing tools.
I have resharper.
I don't have time :). I don't want to strat learning about factories, mocking, IOC and so on. And I don't want to disturb the code too much (re IOC etc.)
I don't have alota money. I saw a recommendation here for SmartBear's TestComplete and then I saw its $2K price tag and I balked at the price: at $99 I would weep at pay, and you can't beat free :)
So, my question is: "What is the simplest and cheapest way for me to start WPF testing, not necessarily the best colution but something that will provide some benefit at low cost?"
If you want to go to the free route, you can have a look at System.Windows.Automation namespace : http://msdn.microsoft.com/en-us/library/system.windows.automation.aspx
See this article : http://msdn.microsoft.com/en-us/magazine/dd483216.aspx
A free and interesting approach is Approval Tests: http://approvaltests.sourceforge.net/. You would essentially "approve" your UI and then execute tests against your app. If the resulting UI doesn't match up with the approved version then the test fails. The comparison here is based on images of your UI - this clearly has pros and cons when compared to other testing approaches.
This video is an example of using Approval Tests with WPF: http://www.youtube.com/watch?v=Xc_ty03lZ9U&list=PL0C32F89E8BBB5368&index=17&feature=plpp_video
Probably the easiest way is to concentrate on the 30 classes (non GUI) to test. It seems (but I'm not sure) that most functionality is in those 30 classes.
If designed well, the model (those 30 classes) can be tested reasonably easy.
For GUI testing a lot more effort is normally needed.
So if you want to spend less time, concentrate on testing the model.
For testing the model, what you normally do is: write stubs for external components (if needed), set the input parameters (depending on your app) and check if the 'output' is what you expect.