I've successfully set up some tests that run complete use cases against our code base using nancy.testing.
Now I'd like to use the same tests for what we call release tests: We deploy the complete application to a staging server and run the system tests against it. We already use that approach successfully with a WCF application.
Is there an easy way to make this work? I've noticed that Browser and BrowserResponse do not use any abstractions, so I cannot replace them with a SystemTestBrowser or similar.
I could of course abstract the whole nancy.testing package away, but I thought I ask here first if someone knows a simpler approach.
Nancy.Testing does not use any network communication at all. If you can think of a nice abstract then please let us know and we can talk and see if it's something we'd like to get into the package!
Related
My company has a folder called tools... which has about 50 some CLI Tools our support agents use for various troubleshooting and reporting...
Company is getting bigger... giving every rep access to our source code just so they can run the tools is not ideal... Plus things like npm package dependencies happen and it's more maintenance than they want.
Ideally, I would create an internal only website that simply presents a dropdown of all the tools in the /tools folder. The webserver (like Express) would execute the scripts and then redirect the standard output to the screen... The kicker is I need to allow for standard input as well since the tools are somewhat interactive... they get to select choices.
I'm sure there are all kinds of security issues with this and I just want emphasize this would be for internal use only and run by trusted users.
I've seen various terminal emulators and projects like this but looked complicated to make it work for our use case. I really just want to let people run a preset number of commands... I feel like this type of thing should exist and I just haven't stumbled upon it yet.
Alternatively... I've considered refactoring the tools to use something like swagger which would present the options for them to fill out but that too isn't ideal as we have conditional prompts...
You could try to use xterm.js to create browser based terminal that can execute the CLI tools.
You could use socket.io and build a node.js app for specific required commands.
socket.io allows for client/server communications on webpage.
node.js allows for a framework where you can pass commands through.
I'm testing REST API end to end and when I want to check the application behaved.
One is using existing REST API where possible but in other cases I don't have REST API available so have two options:
creating an api for tests purposes or checking in the database the data have changed as expected.
Which one is better and why? Is there any harm in doing db calls from your tests?
Since you want to test the API itself (E2E testing) I would probably go with the route of creating a test DB and working on that. If you are performing this manually you can easily, albeit cumbersomely, reset the DB after each test.
You can also, and I would probably go this route, create a Docker container with a ready to use DB that you would update as needed, along with the development of the API itself across time. You could spawn a new container/DB every time you need to perform testing and this would also ease automation and integration in CI/CD pipelines.
Any of these solutions enable you to test the real API, in a real DB. Providing more accurate test results.
Hope this helps!
Cheers!
In e2e tests or acceptance tests you usually only check the observable behavior, so the behavior the user would be able to see, in your case the user of the API, so in theory you should not need to check the database.
If you really want to check the results in the database I would use a project like xmysql https://github.com/o1lab/xmysql to have an Rest API that can be used to check the DB. Reasons are:
you don't have to maintain your own code that serves API endpoints that are not needed in production. That code might get public and might become a security issue
you don't need to mock any database or calls, you can test what you fly, and fly what you test
its easy to test if the system where you run the tests and the system-under-test are different computers. You can test all remotely without needing to open ports to you DB. e.g. its easy to use docker to set it all up, run the tests and then deploy. The only thing the deployment does not have is xmysql
I want to know, if there is any build tool for client side code like angularjs,html5 and other techs.
Like we are using Jenkins for server side code to trigger build.
Since these build tools notify us of failing and passing similarly I need for frontend code.
You might want to take a look at Grunt. This is probably what you are looking for.
I'm using appengine-magic to set up a web application, more or less as described at http://www.digitalbricklayers.com/2012/03/geotasklist-in-jquery-mobile-and.html. The example works on my local machine, locations and tasks are added to a local datastore etc.
My question is if it is possible to interact with the datastore from within a REPL, e.g. call (ds/save! ...) etc. during interactive development? I ask because when I try I get:
NullPointerException No API environment is registered for this thread.
com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppId
(DatastoreApiHelper.java:108)
I'm getting this error no matter if I use an eclipse+counterclockwise based setup or an emacs+slime based setup.
Thanks,
Joachim
There's a bunch of ways to do this.
The easiest way is to go to the admin console (http://localhost:/_ah/admin) and click on the "Interactive Console".
I use django-nonrel, which comes with a command to launch an interactive shell (manage.py shell). If you're not using django-nonrel though, getting it set up though, is somewhat involved. I suspect most of what's necessary is found in the setup_env() function in django-nonrel: https://github.com/django-nonrel/djangoappengine/blob/develop/djangoappengine/boot.py
Getting it all to work is a pain, good luck.
The solution I use 99% of the time is to use pdb and force the interpreter to break at a certain point in my app where I need to do some debugging. See this for instructions: http://eatdev.tumblr.com/post/12076034867/using-pdb-on-app-engine
appengine-magic lets you use App Engine services (like the datastore) as long as the application is running; see https://github.com/gcv/appengine-magic#app-engine-services — as long as you ae/start your application, it should work.
We are developing some testing infrastructure and I have hit a coders block (lack of sleep?)...this seems like it would be a solved problem but I haven't found what I'm looking for via google.
I would like to automatically push builds from our CI server (TeamCity) to a number of machines (growing, but currently 30). These are several WinForms apps and a number of dlls. Once deployed, I would like to kick off tests (NUnit, for both unit and integration tests) and report all results (back to CI? or somewhere else? Not sure).
The target machines are a number of platforms (Win7,Vista, XP, Server 2k8, Server 2k3, Ubuntu, Fedora, Suse, x64, x86, maybe macs down the line)
This gets me part way there (the actual push). But I can't find existing solutions for 'push starting' the tests and reporting back. So far I am thinking of combining the link (or similar) with custom code running on each client machine that watches the deploy directory, runs the tests and reports the results.
Does anyone know of existing solutions?
Links?
Done something similar and care to share?
Edit
If possible, we prefer .net based solutions, but it isn't strictly necessary. I would have tagged the question as such, but ran out of tags :)
You could use KwateeSDCM to both push and start on all the platforms you mention, including mac. However, you'll have to do some coding to get reports out. I'm not familiar with TeamCity but maybe you could push a script along with your application which could then transfer the test results via ftp to a server accessible by TeamCity.
Have a look at: STAF (Software test Automation Framework)
The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring).
Which includes STAX:
STAX is an execution engine which can help you thoroughly automate the distribution, execution, and results analysis of your testcases.
And there's an article here:
http://agiletesting.blogspot.com/2004/12/stafstax-tutorial.html
Assuming you have the push part done already, and you don't mind using a TeamCity license, you can create a TeamCity Command Line Runner build configuration or NUnit test configuration that kicks off the tests on a properly configured agent. The build trigger for this test config would be successful completion of the application build.
So far I have ended up using a seperate build step in TeamCity that executes a bat script that in turn fires of tasks to the list of machines using PsExec. So far my trial runs it is working ok, though I now need to parallelize the copying of build output...
Thanks for the input to those who have provided it.