I am novice in cakephp.Please explain me the concept to shells in cake php and how it is helpful in web development?
I have read http://book.cakephp.org/2.0/en/console-and-shells.html.But still I am not getting the idea of shells in relevance to web development.
Shells are relevant. You can basically write any shell command and script you need. There are other shells like the migration shell coming with the migrations plugin.
I've seen silly attempts were people used cron and wget to call an URL to execute a task every X minutes. That's the perfect example where a shell is the proper solution.
There are plenty of use cases for shells, queuing (emails for example), data conversion, data import... Everything that runs for a long time or checks something like the queuing can be done as a shell. Shells can be also utility or development tools as well. You can even control with the "nice" command how much CPU load a program is allowed to use.
So for example if you have audio or video conversion after upload this should run in the background. The shell task will look for new uploads and when it finds some convert the data in the desired format and never use up more than 20% CPU load for example and by this it won't make the site unresponsive by using 100% CPU load.
If you are new to CakePHP, I'll suggest to check 'bake' console command. ( http://book.cakephp.org/2.0/en/console-and-shells/code-generation-with-bake.html ).
It can help you to generate models, controllers, views, migrations and many others.
It's a powerful tool that will save you a lot of time. When you are generating the model using 'bake' command, it will automatically detect all the tables in your database, work out the database relationship and you can also define the validation for each field.
Here is video tutorial on YouTube http://www.youtube.com/watch?v=Kfu58OozDrM
Related
We're designing a system for a client where they are allowing authenticated users to upload images. We've created an API to upload the files but the client only wants the latest file and delete all previous ones so that there would only ever be one.
We've looked through the docs and can't come across a way for ADAM to handle this in both 2SXC and DNN's file system.
Internally when deleting images we see API calls like the following to the internal 2SXC API, but we're wondering if this is exposed somewhere within the public API?
https://somedomain.com/api/2sxc/app/auto/data/61393528-b401-411f-a001-f423ea46700a/b7d04e2c-c565-496c-8efb-aa133cf90d33/Photo/delete?subfolder=&isFolder=false&id=189&usePortalRoot=false&appId=3
We could probably use the same endpoint above, but we'd likely run into permission issues or changes to the APIs that could be problematic.
Thank you for any advice you can give! Perhaps #iJungleBoy can provide some thoughts on this.
As a solution from a completely different direction, if you are on the later release of 2sxc (v12.8+, v13+), and comfortable programming in C#, you might consider doing this as a "cleanup" from a Dnn Scheduled Task. This can be done with a relatively easy setup. We have a Gist in place that we use as a starter. You simply put the code in the /App_Code folder then setup a normal Dnn Scheduled Task. NOTE that you can scroll down to the first comment on the Gist to see a screenshot of a complete working setup.
Accuraty's AccuTasks template on GitHub Gists
There are two more key things to note:
You need to install Dnn's CodeDom 3.6 because the example uses the later versions C#'s string interpolation - OR remove the few $"ASL2021 - {this.GetType().Name}, Task Scheduled Email", bits or convert to string.Format() or something.
Since your task's code is NOT running in a (2sxc) module, if needed, you'll do stuff like this: 2sxc Docs - Use 2sxc Instance or App Data from External C# Code
So, if you are comfortable writing code that "finds and deletes stuff older than NN days" - this might be the way to go.
I have code that uses Entity Framework to treat data (retrieves data from multiple tables then performs operations on it before saving in a SQL database). The code was supposed to run when a button is clicked in an MVC web application that I created. But now the client wants the data treatment to run automatically every day at a set time (like an SSIS package). How do I go about this?
But now the client wants the data treatment to run automatically every day at a set time (like an SSIS package). How do I go about this?
In addition to adding a job scheduler to your MVC application as #Pac0 suggests, here are a couple of other options:
Leave the code in the MVC project and create an API endpoint that you can invoke on some sort of schedule. Give the client a PowerShell script that calls the API and let them take it from there.
Or
Refactor the code into a .DLL or copy/paste it into a console application that can be run on a schedule using the Windows Scheduler, SQL Agent or some other external scheduler.
You could use some tool/lib that does this for you. I could recommend Hangfire, it works fine (there are some others, I have not tried them).
The example on their homepage is pretty explicit :
RecurringJob.AddOrUpdate(
() => Console.WriteLine("Recurring!"),
Cron.Daily);
The above code needs to be executed once when your application has started up, and you're good to go. Just replace the lambda by a call to your method.
Adapt the time parameter on what you wish, or even better: make it configurable, because we know customers like to change their mind.
Hangfire needs to create its own database, usually it will stay pretty small for this kind of things. You can also monitor if the jobs ran well or not, and check no the hangfire server some useful stats.
So I'm learning about Spark and I have a question about how client libs works.
My goal is to do some sort of data analysis in Spark, telling it where are the data sources (databases, cvs, etc) to process, and store results in hdfs, s3 or any kind of database like MariaDB or MongoDB.
I though about having a service (API application) that "tells" spark what I want to do. The question is: Is it enough setting the master configuration with spark:remote-host:7077 at context creation or should I send the application to spark with some sort of spark-submit command?
This completely depends on how your environment is set up, if all paths are linked to your account you should be able to run one of the two commands, to efficiently open the shell and run test commands. The reason to have a shell, is this will allow you to dynamically run commands and validate/learn how to run/tether commands onto one another and see what results come out.
Scala
spark-shell
Python
pyspark
Inside of the environment, if everything is linked to Hive tables you can check the tables by running
spark.sql("show tables").show(100,false)
The above command will run a "show tables" on the Spark-Hive-Metastore Catalogue and will return all active tables you can see (doesn't mean you can access the underlying data). The 100 means I am going to look at 100 rows and the false means to show the full string not the first N many characters.
In a mythical example if one of the tables you see is called Input_Table you can bring it into the environmrnt with the below commands
val inputDF = spark.sql("select * from Input_Table")
inputDF.count
I would heavily advise, while your learning, not to run the commands via Spark-Submit, because you will need to pass through the Class and Jar, forcing you to edit/rebuild for each testing making it difficult to figure how commands will run without have a lot of down time.
I'm running Oracle 11g SE1 .
Just wondering if there're any tools that would allow me to test the data integrity of a ( mostly read-only ) schema. Essentially, what I want to do is to have some queries that run every night or so and see if they return the expected result. For example:
SELECT COUNT(*) FROM PATIENTS WHERE DISEASE = 'Clone-Killing Nanovirus';
Expected result : 59.
How do people normally do such testing ?
I've used SQLUnit and written about it here. I don't believe any new development is being done on it but it should accomplish your goal.
SQL Developer (free, as in beer) also has a Unit Testing framework. I have installed it and that's about it. I want to use it more, but I've been working with BI the past few years so no external pressure to learn it.
The tests that you want to create sound pretty simple, so either of those should work well for you. The next step would be to have them run on a schedule (cron, windows scheduler, etc) or you can go crazy with a continuous integration tool like Atlassian's Bamboo (haven't used it).
Of course you could skip the tools altogether and just write up scripts that are called from the command line. Fancy would have you writing the results to a database table so you can easily skin it, simple would be piping the results to a text file and reviewing that each day.
Hope this helps.
You could batch up your queries and run a simple perl script using DBI that would run the queries and check them against an accepted tolerance and email you if something didn't meet thresholds. I know I have written such db checking code before to make sure items were within thresholds. Perl is a good tool for this sort of thing as the DBI module can connect to your database and then you can run some canned queries and easily send yourself an email using the MIME package. http://www.perl.com/pub/1999/10/DBI.html
We want to test a connection to an application as a feature of a program we are developing, but to go further with this, we want to actually do a sort of diagnostic test to ensure that the app is working and not just take the service status as gospel (the main windows service running does not mean the app is working fully). However, this app has no api exposed by it, and the forms may be designed in C++ as the app is a mix of many languages (C# is just one of them).
One way to do this is by UI automation and then programatically perform the necessary UI actions to test the app works by performing a fundamental action which uses all the prerequisites like a domain-joined account, etc. However, is there a way to do this non interactively so the forms of the app don't actually show up? If not, is there another way to solve this problem?
Thanks
With no exposed API, you are stuck with automation.
Take a look at autoit. It excels at doing these types of tasks. If it's vbesque script isn't for you it has a DLL interface for use in your favorite language. It is free.
Check it out.
Here are some ideas
Headless UI
You should investigate if this application can be run in a "headless" mode, i.e. without a visible UI. Alot of applications have this option even though it might not always be obivous.
UI Automation
Some tools for UI Automation:
* Microsoft UI Automationbr
* HP QuickTest Proffesional
* AutoIt v3
Analyis Log, If there is one
You could investigate if the application you are connection to writes a log.
* 14:14 Status:OK Activity:Routed 24 messages (or whatever it does) Uptime:2h12m
* 14:15 Status:OK Activity:No Activity Uptime:2h13m
* 14:16 Status:OK Activity:Routed 12 messages, 2 failed see error.log for details Uptime:2h14m
If it does, then you can write a diagnostics script that reads the log, analyses Status, Activity, Uptime and raise flags for any strange behaviour.
Hope this helps!