I have an issue where I'm creating a greenfield web application using ASP.NET MVC to replace a lengthy paper form that manually gets (mostly) entered into an existing SQL Server 2005 database. So the front end is the new part, but I'm working against an existing moderately normalized schema. I can easily add new tables, views, etc. to the schema, but modifying tables is going to be near impossible. There's currently at least 2 existing applications (that I'm aware of) that reference this schema and I've stumbled upon at least a dozen "SELECT * FROM..." statements in each. They exist both in code and in views/triggers/stored procs/etc. That's why modifying existing table schemas is a no-go.
All that being said, the form targets different fields in multiple tables in database. It also has to be dynamic enough to allow the end users to add new questions targeting fields. The end users have a rough idea of the existing database schema so they're savvy enough to know how to pick out tables/fields to be targeted.
I'm have a really rough idea of how I could tackle this, but it seems like complete overkill and will be difficult to write up. I'm hoping somebody might have a simple(r) way of handling this sort of project that I haven't thought of.
If users know DB schema maybe you should go with Dynamic Data project and just create a web app front end of that DB to them. So you would only make the model they need and do the application that will display data from those tables with insert/edit capabilities.
But it's completely different story if they have some additional functionality to it.
Related
I'm using SQL Server 2019. I have a "MasterDB" database that is exposed to GUI application.
I'm also going to have as many as 40 user database like "User1DB", "User2DB", etc. And all these user databases will have "exact same" schema and tables.
I have a requirement to copy tables data (overwriting target) from one user database (let's say "User1DB") to the other (say "User2DB"). I need to do this from within "MasterDB" database since the GUI client app going to have access to only this database. How to handle this dynamic situation? I'd prefer static SQL (in form of Stored Procedures) rather than dynamic SQL.
Any suggestion will greatly be appreciated. Thanks.
Check out this question here for transferring data from one database to another.
Aside from that, I agree with #DaleK here. There is no real reason to have a database per user if we are making the assumption that a user is someone who is logging into your frontend app.
I can somewhat understand replicating your schema per customer if you are running some multi-billion record enterprise application where you physically have so much data per customer that it makes sense to split it up, but based on your question that doesn't seem to be the case.
So, if our assumptions are correct, you just need to have a user table, where your fields might be...
UserTable
UserId
FName
LName
EmailAddress
...
Edit:
I see in the comments you are referring to "source control data" ... I suggest you study up on databases and how they're meant to be designed, implemented, and how data should be transacted. There are a ton of great articles and books out there on this with a simple Google search.
If you are looking to replicate your data for backup purposes, look into some data warehouse design principles, maybe creating a separate datastore in a different geographic region for that. The latter is a very complex subject to which I can't go over in this answer, but it sounds like that goes far beyond your current needs. My suggestion is to backtrack and hash out the needs for your application, while understanding some of the fundamentals of databases (or different methods of storing data). Implement something and then see where it can be expanded upon / refactored.
Beyond that, I can't be more detailed than the original question you posted. Hope this helps.
I looked around for this task I have on my hands but did not find anything helpful. I am primarily a Java person with sound knowledge of database from software development point of view. I do have some knowledge of DBA functions with what can and cannot be done but not able to come up with a good solution.
The task I have is to compare the databases created in SQL Server and Oracle by our application installer.
I think I have been able to come up with some queries (of course, by searching online) in SQL Server that will give me things like number of tables in a schema, each table's columns with data types and indexes, different types of constraints, triggers, etc. (with their count) created for each of those tables. I can provide those SQLs if somebody is interested.
However, Oracle seem to be more tricky. I would appreciate if somebody can help or maybe point me in the right direction.
I am trying to find out somethings like following:
Number of tables created
Number of indexes, constraints (with their types), triggers for each of those tables
Number of stored procedures/functions created
Number of views created
Any help will be greatly appreciated.
Thank you.
First off, if you are already comfortable writing Java code, I'm not sure that I would be writing a bunch of SQL to do this comparison. JDBC already has a DatabaseMetaData class that has methods like getTables to get all the tables. That would give you one API to work with and let you leverage the fact that the folks that wrote the JDBC drivers already wrote all the code to query the data dictionary tables in whatever database you are using. This will also let you focus on differences in how the objects your installer creates will be perceived by the application.
If you are going to write specific SQL, the Oracle data dictionary tables are pretty easy to work with. The ones you'll care about are going to follow the pattern [user|all|dba]_<<type of thing>>. The [user|all|dba] prefix indicates whether you are looking for objects that you own (user), objects that you have access to (all), or all objects in the database (dba). Normal users often don't have access to the dba views because that is a potential security issue-- generally you don't want people to know that an object exists if they don't have access to it. In my examples, I'll use the all versions of the objects but you can change all to user or dba depending on what you're after.
all_tables will show you information about all the tables you have access to. You probably want to add a filter on owner for the schema(s) that your installer touches since you may have access to tables that are not part of your application.
all_indexes, all_constraints, and all_triggers will show you information about indexes, constraints, and triggers. Again, you may want to add a predicate on owner to limit yourself to the schema(s) that you care about.
all_procedures will show you information about procedures and functions both stand-alone and in packages.
all_views will show you information about all views.
If you are really just interested in counts, you may be able to simply go to all_objects and do a count grouping by object_type. I'm guessing that you'll want to see attributes of the various objects so you'll want to go to the various object-specific views.
I'm planning to develop a web application in CakePHP that shows information in graphics and cards. I chose CakePHP because the information that we need to show is very structured, so the model approach makes easier to manage data; also I have some experience with MVC from ASP.NET and I like how simple is to use the routing.
So, my problem is that the multiple organizations that could use the app would have their own database with a different schema that the one we need. I can't just set their string connection in the app.php file because their database won't match my model.
And the organization datasource couldn't fit my model for a lot of reasons: the tables don't have the same name, the schema is different, the fields of my entity are in separated tables, maybe they have the info in different databases or also in different DBMS!
I want to know if there's a way to make an interface that achieves this
In such a way that cakephp Model/Entity can use data regardless of the source. Do you have any suggestions of how to do that? Does CakePHP have an option to make this possible? Should I use PHP with some kind of markup language like JSON or XML? Maybe MySQL has an utility to transform data from different sources into a view and I can make CakePHP use the view instead of the table?
In case you have an answer be as detailed as you can.
This other options are possible if it's impossible to make the interface:
- Usw another framework that can handle this easier and has the features I mentioned above.
- Make the organization change their database so it matches my model (I don't like this one, and probably they won't do it).
- Transfer the data in the application own database.
Additional information:
The data shown in graphics are from students in university. Any university has its own database with their own structure and applications using the db, that's why isn't that easy to change structure. I just want to make it as easy as possible to any school to configure their own db.
EDIT:
The version is CakePHP 3.2.
An important appointment is that it doesn't need all CRUD operations, only "reading". Hope that makes the solution easier.
I don't think your "question" can be answered properly, it doesn't contain enough information, not enough details. I guess there is something that will stay the same for all organizations but their data and business logic will be different. But I'll try it.
And the organization datasource couldn't fit my model for a lot of reasons: the tables don't have the same name, the schema is different, the fields of my entity are in separated tables, maybe they have the info in different databases or also in different DBMS!
Model is a whole layer, so if you have completely different table schemas your business logic, which is part of that layer, will be different as well. Simply changing the database connection alone won't help you then. The data needs to be shown in the views as well and the views must be different as well then.
So what you could try to do and what your 2nd image shows is, that you implement a layer that contains interfaces and base classes. Then create a Cake plugin for each of the organizations that uses these interfaces and base classes and write some code that will conditionally use the plugin depending on whatever criteria (guess domain or sub-domain) is checked. You will have to define the intermediate interfaces in a way that you can access any organization the same way on the API level.
And one technical thing: You can define the connection of a table object in the model layer. Any entity knows about it's origin but you should not implement business logic inside an entity nor change the connection through an entity.
EDIT: The version is CakePHP 3.2. An important appointment is that it doesn't need all CRUD operations, only "reading". Hope that makes the solution easier.
If that's true either use the CRUD plugin (yes, you can use only the R part of it) or write some code, like a class that describes the organization and will be used to create your table objects and views on the fly.
Overall it's a pretty interesting problem but IMHO to broad for a simple answer or solution that can be given here. I think this would require some discussion and analysis to find the best solution. If you're interested in consulting you can contact me, check my profile.
I found a way without coding any interface. In fact, it's using some features already included in the DBMS and CakePHP.
In the case that the schema doesn't fit the model, you can create views to match de table names and column names from the model. By definition, views work as a table so CakePHP searches for the same table name and columns and the DBMS makes the work.
I made a test with views in MySQL and it worked fine. You can also combine the data from different tables.
MySQL views
SQL Server views.
If the user uses another DBMS you just change the datasource in app.php, and make the views if it's necessary
If the data is distributed in different DBMS, CakePHP let's you set a datasource for each table, you just add it to app.php and call it in the table if it's required.
Finally, in case you just need the "reading" option, create a user with limited access to the views and only with SELECT privileges.
USING:
CakePHP 3.2
SQL SERVER 2016
MySQL5.7
Im wondering what will be the best way to organize my DB. Let me explain:
Im starting a new "big" project. This big project will be composed by few litle ones. In general the litle projects are not related to each other, they are just features of the big one.
One thing that all the projects have in common is the users that are going to use it.
So my questions are:
Should i create different DB for each one of the litle projects
(currently each project will contain 4-5 tables)
How to deal with the users? Should I create one DB for all the users
or should i
duplicate the users table in every DB? Have in mind that the
information about the users is used a lot in every litle project,
it's NOT only for identification purposes.
Thanks in advance for your advice.
This greatly depends on the database you choose to use.
If these "sub-projects" are designed to work as one coherent unit, then I strongly recommend you keep it all in the same database. One backup, one restore, one unit.
For organizational purposes, if you are using a database which supports it, select a different Schema per project. PostgreSQL and SQL Server are two databases (among others) which support this effortlessly.
In the case of a database like MySQL, I recommend you pick a short prefix for each subproject and prefix all tables accordingly. "P1_Customer" for example.
Shared data would go in it's own schema or prefix, like Global or something like that.
Actually, this was one of the many reasons we switched our main database from MySQL to PostgreSQL. We've been heavy users of both, and I really appreciate the features that PostgreSQL offers. SQL Server, if you are in a windows environment, is a great database IMO as well.
If the little projects are "features of the big one" then I don't see a reason why you wouldn't want just one user table for the main project. The way you setup the question makes this seem true "If there is a user A in little project 1, then there must be a user A in the 'big' project." If that is true, you should likely have the users in the big db instead of doing duplication unless you have more qualifying details.
i think the proper answer is 'it depends'.
Starting your organization down the path of single centralized system is good on many levels. I think in general i would recommend this.
however:
if you are going to have dramatically different development schedules, or dramatically different user experiences with the various sub projects, then you may be better off keeping them separate.
I'd have a look at OpenID or some other single sign-on protocol depending on the nature of your application. OpenID includes a mechanism called "attribute exchange", which allows applications to retrieve profile information from the OpenID provider.
This allows you to create a central user profile repository, with an authentication scheme, and have your individual apps query that repository for profile information.
The question as to how to design your database is hard to answer without more information. In most architectures, "features" within an application tend to be closely linked - "users" are related to "accounts" are related to "organisations" etc.
I'd recommend looking at the foreign key relationships to answer this question. If you have lots of foreign keys, build a single database for all tables. If you have "clusters" of foreign keys, and you want to have a different life cycle for each application (assuming the clusters map neatly to the applications), consider separate databases.
By "life cycle", I mean mostly the development lifecycle - app 1 might deploy weekly, app 2 monthly, app 3 once only and then be frozen.
I'm pretty new to database development in general and I've never used ORM before. I'm interested in the benefits of using one, specifically saving time writing boilerplate SQL queries. I'd like to use ORM for a project that I'm working on right now, but I'm not sure it's applicable.
This project is more akin to change tracking for very small (<= 500 characters) documents. I need to track edits and categorizations made by multiple users. Not really to see the specific changes they make, but more to see if the users agree with each other. I am using a SQL database for this (as opposed to actual document control) for a few reasons:
The documents are really small; and I'm only interested in the strings, not really in files.
I wanted the ability to perform ad-hoc queries against the data for development purposes, and didn't want an unpleasantly surprised halfway through that a particular document control package couldn't do what I wanted.
From most of what I've read it seems like you need a direct mapping from columns to data fields in an object to use ORM. What I have now does not even come close to this. To create objects representing documents in different stages of editing I have to cobble together data from columns in different tables, in different combinations.
So my question is: Does an ORM like Hibernate apply to this type of project? And if it does can one be added to an existing application/database?
If it makes a difference: I'm using Java, MySQL, and JDBC. The web app users have access to for edits is made with GWT and hosted via Tomcat6. If I need it, I have complete control of the webserver.
Thanks.
Does an ORM like Hibernate apply to
this type of project?
Yes
And if it does can one be added to an
existing application/database?
Yes
My opinion is that an ORM tool could be useful for you but you really need to delve into it to see for yourself. Remember when you use an ORM tool you are not forced to use only that to connect to your database. ORM tools in general make the most sense for applications that store data in a very object like structure. For instance your user code might be the place to start. Usually you only create 1 user at a time, you edit 1 user at a time, you check if 1 user is logged into. It also makes sense for things where you would return a list of results like Order Lines. Where I have run into issues with ORM tools is when you have complex data that requires multiple joins especially back to the table you started in. For those cases you might want to keep doing what you are doing. Overall, ORM tools are great but they are like a lot of other things in software development. Try them out on a small part of your code and use them where they work and don't where they don't. Ultimately, you are the one that will have to deal with and maintain whatever you make. Just educate yourself on Hibernate and I am sure you will know what to do!
I think that ORM (I would suggest using the JPA standard, probably with Hibernate as the provider) could suit your project.
It is fairly traditional, as you say, for database columns to map directly onto object fields. If you need to keep your existing database structure (which apparently doesn't map at all well to your objects), then you might find that its more trouble than it's worth to use ORM.
While it's certainly possible to use ORM to map to a specific database schema - perhaps because it's used by other systems - my view is that one of the biggest advantages of ORM is that you can almost ignore the schema. Once you design your objects, and tell hibernate about them, hibernate will create whatever tables it needs.