I am trying to setup the RoundHousE project in my application to handle the database migration and version handling. I am following this article. It is fine as far as I know the database name exactly.
But I am not able to find, how should I handle the dynamic name of databases, because in my application I have separate database for each client, and list of these databases in a table in my main database. So name goes like: client1_db, client2_db etc.
Any solution or pointer towards the solution will be a great help.
A pointer towards the solution - The wiki https://github.com/chucknorris/roundhouse/wiki
Asking about passing the database name dynamically is a bit weird to me as when you run rh.exe or use the embedded DLL, database name is one of the required arguments. So you always have to pass the name dynamically. See https://github.com/chucknorris/roundhouse/wiki/ConfigurationOptions#main-stuff
Reading and trying to understand what you are asking, it seems you have a list of database names in a main database somewhere that you want to give to RoundhousE? To do that you would need to create something custom that can gather the name of the database(s) you are looking for and provide the result to RoundhousE.
Related
I have a email folder in Outlook that contains 100s of emails which record my discussions with a developer of some bespoke software. I want to import these into SQL to create a knowledge base of information that can be searched upon to extract all the decisions that we have made during the course of the 2 year project.
Having sreached the net, I found that it is very easy to dump the contents of an email folder into Access using the import data functionality. In fact I have linked the table and so believe (never used Access before!!) that I now have an Access table that is connected in 'real-time' to the Outlook folder. This is eactly what I want BUT in SLQ as this is something that I am very familiar with using.
So I have tried to import the Access database into SQL (which also appears to be relatively easy) but keep getting the message that 'The source database ...contains no visible tables or views'. Checking SQL pemissions, I am owner of this new databse.
Two questions please. First, cant believe that going through Access is the simplest way to do this and presume that I will loose the 'real-time' link - am I right? Second, given that I can see my Access database has a visible table, why am I getting the error?
The easiest and quickest way is to create a VBA macro where you can populate your SQL database from Outlook emails. You can build the table structure according to your needs and extract the required information from Outlook using VBA. I'd suggest processing emails in chunks using the Find/FindNext or Restrict methods of the Items class, so you will not reach the reference counter limit. The MailItem properties you may find described in MSDN.
BTW The internal store (if you use the cached mode) in Outlook acts like a database. So, why do you need to introduce yet a new database?
I've worked a lot with Pentaho PDI so some obvious things jump out at me.
I'll call Connection Managers "CMs" from here on out.
Obvious, Project CMs > Package CMs, for extensability/ re-usability. Seems a rare case indeed where you need a Package-level CM.
But I'm wondering another best practice. Should each Project CM itself be composed of variables? (or parameters I guess).
Let's talk in concrete terms. There are specific database sources. Let's call two of them in use Finance2000 and ETL_Log_db. These have specific connection strings (password, source, etc).
Now if you have 50 packages pulling from Finance2000 and also using ETL_Log_db ... well ... what happens if the databases change? (host, name, user, password?)
Say it's now Finance3000.
Well I guess you can go into Finance2000 and change the source, specs, and even the name itself --- everything should work then, right?
Or should you simply build a project level database called "FinanceX" or whatever and make it comprised of parameters so the connectoin string is something like #Source + # credentials + # whatever?
Or is that simply redundant?
I can see one benefit of the parameter method is that you can change the "logging database" on the fly even within the package itself during execution, instead of passing parameters merely at runtime. I think. I don't know. I don't have a mountain of experience with SSIS yet.
SSIS, starting from version 2012, has SSIS Catalog DB. You can create all your 50 packages in one Project, and all these packages share the same Project Connection Managers.
Then you deploy this Project into the SSIS Catalog; the Project automatically exposes Connection Manager parameters with CM prefix. The CM parameters are parts of the Connection Manager definition.
In the SSIS Catalog you can create so called Environments. In the Environment you define variables with name and datatype, and store its value.
Then - the most interesting part - you can associate the Environment and the uploaded Project. This allows you to bind project parameter with environment variable.
At Package Execution - you have to specify which Environment to use when specifying Connection Strings. Yes, you can have several Environments in the Catalog, and choose when starting Package.
Cool, isn't it?
Moreover, passwords are stored encrypted, so none can copy it. Values of these Environment Variables can be configured by support engineers who has no knowledge of SSIS packages.
More Info on SSIS Catalog and Environments from MS Docs.
I'll give my fair share of experience.
I recently had a similar experience at work, our 2 main databases name's changed, and i had no issue, or downtime on the schedules.
The model we use is not the best, but for this, and for other reasons, it is quite confortable to work with. We use BAT files to pass named parameters into a "Master" Job, and basically depending on 2 parameters, the Job runs on an alternate Database/Host.
The model we use is, in every KTR/KJB we use a variable ${host} and ${dbname}, these parameters are passed with each BAT file. So when we had to change the names of the hosts and databases, it was a simple Replace All Text Match in NotePad++, and done, 2.000+ BAT Files fixed, and no downtime.
Having a variable for the Host/DB Name for both Client Connection and Logging Connection lets you have that flexibility when things change radically.
You can also use the kettle.properties file for the logging connection.
I will preface this right off the bat by saying that I am new to database design.
I have been working to rewrite some legacy code that controls an import process into one of our pieces of software. Part of this new process includes the modification of the incoming XML files (that come into our system via FTP) to remove certain elements as well as swap values in special cases.
As a part of the new system, we are implementing a way of versioning inside the database so that we can pull the most recent version of the xml directly from that instead modifying the file over and over again. In order to prove that this can be done, I have created a very simple table inside of SQL Server 2016 that stores the XML, then wrote a simple PowerShell script to pull that XML file from the database and store it inside of an object. Now that I know that this is indeed possible, I need to refine how I design the table.
This is where my expertise starts to take a hit. As of right now, the table contains three columns: xml_Version, xml_FileID, and xml_FileContents.
The general idea is to have a GUID (xml_FileID) that is tied to each version of the XML and another column that indicates what version of the XML that is. I would also assume that you need some way of tying each version of the XML to it's original file, too.
I was hoping that someone could point me in the right direction about how I should go about designing the table to accomplish this task. I can provide more information if needed.
Thanks.
Edit: I think what I'm having the most trouble grasping is what I should be referencing when I'm trying to grab data out of the database. Storing the XML in the table with a unique identifier is the easy part - but the unfortunate part is that there's nothing in the XML itself that I can grab out of there that would be able to uniquely identify the correlating data within the database. Does that make sense?
I am construction a login system for Yii2 that can create the identity from either within a table in the Yii2 application or from data from an external database, the data could reside in one of many, many databases.
I know i could setup second, third, fourth databases in the apps config, but it doesn't fit the use case as, as the database belongs to a user who could change it, etc.
What i need to do it instantiate an ActiveRecord model passing it the database details of the database it should connect to, to query.
I have managed to very much confuse myself over this, in trying to work out the correct way to do it.
I know you can pass a config array to the constructor, but am unable to understand how i should do it, should i create and object of \yii\db\connection and pass it to the __construct?
If so how do i then perform the connection ?
I have searched all over for similar use case, but am unable to discover anything that helps, even a pointer would be great.
Many Thanks
Ok, so after so more head banging and searching in a different way, i have discovered a very simple and sensible approach, and the answer does come from SO.
Yii2 set db connection at runtime
I'd like to know your approach/experiences when it's time to initially populate the Grails DB that will hold your app data. Assuming you have CSVs with data, is is "safer" to create a script (with whatever tool fits you) that:
1.-Generates the Bootstrap commands with the domain classes, run it in test or dev environment and then use the native db commands to export it to prod?
2.-Create the DB's insert script assuming GORM's version = 0 and incrementing manually the soon-to-be autogenerated IDs ?
My fear is that the second approach may lead to inconsistencies for hibernate will have the responsability for the IDs generation and there may be something else I'm missing.
Thanks in advance.
Take a look at this link. This allows you to run groovy scripts in the normal grails context giving you access to all grails features including GORM. I'm currently importing data from a legacy database and have found that writing a Groovy script using the Groovy SQL interface to pull out the data then putting that data in domain objects appears to be the easiest thing to do. Once you have the data imported you just use the commands specific to your database system to move that data to the production database.
Update:
Apparently the updated entry referenced from the blog entry I link to no longer exists. I was able to get this working using code at the following link which is also referenced in the comments.
http://pastie.org/180868
Finally it seems that the simplest solution is to consider that GORM as of the current release (1.2) uses a single sequence for all auto-generated ids. So considering this when creating whatever scripts you need (in the language of your preference) should suffice. I understand it's planned for 1.3 release that every table has its own sequence.