Related
I have 2 simple GUI-test on a WPF application. They work alright from VisualStudio.
Now I am trying to also do that in my Azure pipeline. Ik keep on bumping against the same error right at the start:
"OpenQA.Selenium.WebDriverException: An element could not be located on the page using the given search parameters"
Basically I am working along this line: WinAppDriver in CI with Azure Pipelines. One difference with the example is that I have put my tests in my build pipeline instead of my release pipeline. As far as I can determine that should not be the cause of my problem, using the same Windows-2019 agent.
This is how the pipeline looks like, up to the tests:
In the past I have had problems from VisualStudio similar to those decribed here, but I have left those behind. FindElementByName - Element couldn't be located
Nevertheless I have tried to improve on these aspects by using DefaultWait. It works fine from VisualStudio but has not helped me on Azure.
I read a couple of times about an 'interactive mode' for the agent. But I don't see anything about that in the guideline, and it seems to apply to self hosted agents. I also don't see any configuration option on the standard agent. So I am confused about that.
I am lost here.
Could someone tell me what I am missing?
It would be much appreciated, I have been spending quite some time on this already.
This took me a while to find the necessary information and piece together the solution, which turned out to work.
The key thing is using an interactive agent, which has to be self hosted. Key instruction for me was following Self-hosted Windows agents. This resulted in installing an environment on my own hardware, and to be pretty simple and straight forward.
This implies one has to startup this environment with Powershell, and probably guard the graphical test as it executes. For an one man team as mine that is no problem, for a larger team this may be an issue.
An additional advantage is that the entire building and execution environment is now under one's own control for overview and inspection.
It's funny to push a commit and see the pipeline progress on Azure, plus starting up the graphical test on my own hardware.
I was working on the tensorflow object detection API. I managed to train it locally on my computer and get decent results. However, when I tried to replicate the same on GCP, I had several errors. So, basically, I followed the documentation mentioned in the official tensorflow -running on cloud documentation
So this is how the bucket is laid out:
Bucket
weeddetectin-data
Train-packages
This is how I ran the training and evaluation job:
Running a multiworker training job
Running an evaluation job on cloud
I then used the following command to monitor on tensoboard:
tensorboard --logdir=gs://weeddetection --port=8080
I opened the dashboard using the preview feature in the console. But it says no dashboards are active for the current data set.
No Dashboards are active
So, I checked on my activity page to really see if the training and evaluation job were submitted:
Training Job
Evaluation Job
It seems as if there are no events files being written to your bucket.
The root cause could be that the manual your are using refers to an old version of the tensor models.
Please try and change
--train_dir=gs:...
to
--model_dir=gs://${YOUR_BUCKET_NAME}/model
And resend the job, once the job is running check the model_dir in the bucket to see if the files are written there.
Check out: gcloud ml-engine jobs documentation for additional read.
Hope it help!
Are there any Azure Automation DSC gurus who can help with some guidance and know-how for pushing through a couple impasses I am currently encountering?
The task at hand is: Use Azure Automation Runbook to provision a VM. That VM should immediately be associated with a DSC configuration, which will adjust Windows features, settings, and install SQL Server according to a specific configuration. All tasks conducted need to be written in PowerShell and should require no manual input via Azure portal at any point.
At this time, the Runbook provisioning the VM is working perfectly. However, associating this new node with a DSC configuration is still a manual process, which also is working (with the exception of the next issue mentioned below). However, this process needs to be automated instead. How is this done? Via DSC resources as children of the VM resource in the ARM template?
Getting SQL Server installed is the next step. The xSQLServer DSC module seemed perfect for achieving this, but it currently has a bug in Azure Automation, which means that the xSQLServerSetup resource is not available, even when using older versions of xSQLServer. So, there appear to be two possible workarounds to this…
Workaround 1: Not use xSQLServer and just run a PS script that is local on the newly provisioned VM to install SQL Server using a command line installation using an INI file. The PS script to install SQL works, but only when run manually. When attempting to have DSC run this script, Azure is throwing an error that the script is not digitally signed. So, there appears to be a permissions scoping issue at play, and the DSC credential is not able to run the local PS script even though the local admin credential is being passed in. How does one get around this?
Workaround 2: Apparently, it is supposed to be possible to provision a VM, compile the DSC MOF local on that machine (with the full version of xSQLServer), and then push that registration back to Azure Automation. Though, it is unclear how exactly this would be done, as it appears to also require the execution of a local PS script, thus providing the same impasse as the first workaround. Is this perhaps via a Custom Script extension in the ARM template, or…?
I can see all of the parts in play, and I’ve found several helpful resources online that give breadcrumbs to the solution. But, the breadcrumbs are too far apart, and the proper way of wiring everything together is proving to be elusive. So, I’m here humbly asking for help and guidance in getting this worked out.
Any help would be greatly appreciated.
Thanks!
First of all that's a lot of questions instead of 1.
unless this is some kind of homework - there is no point in installing sql on a vm, there are a lot of vm + sql images in Azure and it would take 5 minutes instead of 60 to provision such a vm.
When attempting to have DSC run this script, Azure is throwing an error that the script is not digitally signed. - this means your script is not signed (not related to rights\permissions), look for execution policy, you need to set it to unrestricted before running your script (but you don't need to, because of the first point).
you compile mof or upload it and then you can "tie" a vm to that mof, it can be automated with powershell (both parts), there are a lot of guides on how to do that. Like this
As a general rule, use arm template to do the whole thing, again, there are lots of examples on how to achieve that (just browse this repo). Provisioning infrastructure with powershell (on azure) is not the best way of doing things.
We are developing some testing infrastructure and I have hit a coders block (lack of sleep?)...this seems like it would be a solved problem but I haven't found what I'm looking for via google.
I would like to automatically push builds from our CI server (TeamCity) to a number of machines (growing, but currently 30). These are several WinForms apps and a number of dlls. Once deployed, I would like to kick off tests (NUnit, for both unit and integration tests) and report all results (back to CI? or somewhere else? Not sure).
The target machines are a number of platforms (Win7,Vista, XP, Server 2k8, Server 2k3, Ubuntu, Fedora, Suse, x64, x86, maybe macs down the line)
This gets me part way there (the actual push). But I can't find existing solutions for 'push starting' the tests and reporting back. So far I am thinking of combining the link (or similar) with custom code running on each client machine that watches the deploy directory, runs the tests and reports the results.
Does anyone know of existing solutions?
Links?
Done something similar and care to share?
Edit
If possible, we prefer .net based solutions, but it isn't strictly necessary. I would have tagged the question as such, but ran out of tags :)
You could use KwateeSDCM to both push and start on all the platforms you mention, including mac. However, you'll have to do some coding to get reports out. I'm not familiar with TeamCity but maybe you could push a script along with your application which could then transfer the test results via ftp to a server accessible by TeamCity.
Have a look at: STAF (Software test Automation Framework)
The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring).
Which includes STAX:
STAX is an execution engine which can help you thoroughly automate the distribution, execution, and results analysis of your testcases.
And there's an article here:
http://agiletesting.blogspot.com/2004/12/stafstax-tutorial.html
Assuming you have the push part done already, and you don't mind using a TeamCity license, you can create a TeamCity Command Line Runner build configuration or NUnit test configuration that kicks off the tests on a properly configured agent. The build trigger for this test config would be successful completion of the application build.
So far I have ended up using a seperate build step in TeamCity that executes a bat script that in turn fires of tasks to the list of machines using PsExec. So far my trial runs it is working ok, though I now need to parallelize the copying of build output...
Thanks for the input to those who have provided it.
Let's say we have a continuous integration server. When I check in, the post-hook pulls the latest code, runs the tests, packages everything. What is the best way to also automate the database changes?
Ideally, I'd build an installer that could either build a database from scratch or update an existing one using some automated syncing method.
I've recently bumped into an article, that might be of use.
The author explained some of the best continuous integration practices including testing, processing and automation.
Here are some of the key takeaways:
In many shops code is unit tested at the point of commit. For databases, it is preferred running all unit tests at once and in sequence against a QA database, vs development, as a part of the Test step
The test step is a critical part of any CI/CD process. Test scripts, including unit tests themselves, should also be versioned in source control, extracted at the point of the Build step and executed
Pulling data from production is appealing as a quick expedient, but is never a good idea
The best approach is using a tool or script to quickly, repeatedly and reliably create synthetic test data for your transactional tables
Running unit tests to produce manual summary results for human consumption defeats the purpose of automation. We need machine readable results, that can allow an automated process to abort, branch and/or continue.
Running a CI process, which requires 100% of all tests to pass, is akin to not having CI at all, if the workflow pipeline is set up atomically to stop on failure, which it should. To thread the needle, tests should have built in thresholds, that will raise an error based on either the % of tests failing or in some cases, if certain high priority tests fail.
All processes should ultimately produce a Boolean result of pass or fail, but some non-automated processes can easily find their way into your CI workflow pipeline (e.g. unit testing). Software should be plug-n-play into any workflow pipeline, taking known inputs and producing expected outputs – like pass, fail.
CI/CD process should be aborted on failure and a notification email should be immediately sent vs continuing to cycle the pipeline.
The CI process should not cycle again until any errors in the last build are fixed. On failure, the entire team should get the failure notification, including as many details as to what failed as possible.
If a pipeline takes 1 hour, from start to finish, to complete, including all the testing, then all the build intervals should be set to no less than one hour and all new commits should be queued, and applied to the next build.
No plain text passwords should exist in automation scripts
If you have the opportunity to define and control the whole database management and db creation process, have a serious look at DB Ghost - it's more than just a tool - it's a process.
If you like it and can implement it, you'll get great returns on it - but it's a bit of a "all-or-nothing" kind of approach. Recommended.
I would caution against using a db backup as a development artifact, most CI best practices suggest that you manage the schema, procedures, triggers, and views as first class development artifacts. The side effects is that you can take this one step further and use them to build a new database whenever you want, ideally you also have some data that can be pushed into the database.
Here is a cliff notes version to get your feet wet, but there is lots out there in this space:
http://www.infoq.com/news/2008/02/versioning_databases_series
I like some of the ideas that Scott Ambler has here as well, the site is good but the book is surprisingly deep for such a difficult set of problems.
http://www.agiledata.org/
http://www.amazon.com/exec/obidos/ASIN/0321293533/ambysoftinc
Red Gate is a quite robust solution and it works out of the box.
But the best thing is that you can integrate it with your continuous integration process. I use it with Msbuild and Hudson.
quickly explaining how it works:
http://blog.vincentbrouillet.com/post/2011/02/10/Database-schema-synchronisation-with-RedGate
if you need to know more about this, feel free to ask
The Red Gate approach using SQL Source Control and the SQL Compare Pro command line is detailed with code samples here:
http://downloads.red-gate.com/HelpPDF/ContinuousIntegrationForDatabasesUsingRedGateSQLTools.pdf
Troy Hunt wrote an article on Simple Talk entitled "Continuous Integration for SQL Server Databases":
http://www.simple-talk.com/content/article.aspx?article=1247
Have you looked at FluentMigrator? The default download includes Nant scripts that would be easy to add in to a CI. Free, open source and easy to use. Works for a wide variety of databases.
The latest version (5.0) of DB Ghost doesn't suffer from the "non ASCII character" problem (it just means that the file is UTF8 encoded) and it should be able to do exactly what you need.
Also, the tools can actually be used standalone to perform the various functions (scripting, building, comparing, upgrading and packaging) if you want, it's just that using them all together provides a full end-to-end process thus making the overall value greater than the sum of it's parts.
In essence, to make changes to the schema you update individual object creation scripts and per-table insert scripts (for reference data) that are held under source control just like you were developing a “day one” greenfield database. The DB Ghost tools are used to enable the whole thing by building these scripts into a brand new database (using continuous integration if required) and then comparing and upgrading a target database, which can be a copy of the production database. This process produces a delta script which can be used on the real production database during go-live.
You can even produce a Visual Studio database project and add it into any solutions you currently have.
Malc
I know this post is old, but we have a new solution that takes the following approach:
Developers script individual SQL changes and commit them to source
control.
Our program (OneScript) pulls the change script files from
source control, filters and sorts them, and generates a single
release script file.
That release script file is then applied to a
database to do a release.
Our home page here explains this process in more detail and has a link to an example that does these steps automatically from a Subversion hook. So soon after a commit, the developer receives an email saying if the release was successful or had errors. The PowerScript code is included.
Disclaimer -I'm working at the company that makes OneScript.