I was assigned the task to create a simple Database Management System in a class so I looked up Postgres and noticed that the CLI tool (psql) has commands (\d and \l) that output information about the database and columns of a table in the form of tables like when you do a SELECT. So my question is If Postgres manages user tables inside system tables? and that way when you do \d or \l you are actually doing a SELECT on those system tables. This is just to understand if that would be a good way of managing tables in a database or not and just use regular data structures like lists.
It does indeed. You can run psql with -E to see the queries it is using.
Then check the online manuals
The items to search for are "system catalogs" and "INFORMATION_SCHEMA". The latter is a standard way of describing database schemas and should mostly work between different RDBMS.
Yes, Postgres uses tables that it creates to manage the tables that you create.
There is an entire chapter in the documentation explaining. To quote:
The system catalogs are the place where a relational database management system stores schema metadata, such as information about tables and columns, and internal bookkeeping information. PostgreSQL's system catalogs are regular tables.
As mentioned in the other Answer, the SQL standard requires metadata be provided in some table structures as defined within the standard. These must be housed within a schema named exactly INFORMATION_SCHEMA. Postgres provides that schema and its prescribed tables, but implements them as a view on the actual system tables. See the chapter on INFORMATION_SCHEMA in Postgres documentation.
You can access the metadata, such as to get a list of all the tables you have defined, or get a list of all the columns you defined in a particular table. To do so, perform a query in SQL using SELECT like any other query.
For portability, meaning to write code that works in other database systems in addition to Postgres, query against INFORMATION_SCHEMA.
For additional details not required by the SQL standard, and for Postgres-specific info, query against the Postgres-specific system tables. Their names all start with pg_.
Related
From Database System Concepts, by Silberschatz et al:
4.5.7 Schemas, Catalogs, and Environments
Like early file systems, early database systems also had a single name
space for all relations. Users had to coordinate to make sure they did
not try to use the same name for different relations. Contemporary
database systems provide a three-level hierarchy for naming relations.
The top level of the hierarchy consists of catalogs, each of which can
contain schemas. SQL objects such as relations and views are contained
within a schema. (Some database implementations use the term
“database" in place of the term catalog.)
In order to perform any actions on a database, a user (or a program)
must first connect to the database. The user must provide the user name
and usually, a password for verifying the identity of the user. Each
user has a default catalog and schema, and the combination is unique
to the user. When a user connects to a database system, the default
catalog and schema are set up for the connection; this corresponds to
the current directory being set to the user’s home directory when the
user logs into an operating system.
To identify a relation uniquely, a three-part name may be used, for
example, catalog5.univ schema.course We may omit the catalog
component, in which case the catalog part of the name is considered to
be the default catalog for the connection. Thus if catalog5 is the
default catalog, we can use univ schema.course to identify the same
relation uniquely.
A relation has a schema, which is the collection of all the
attributes of the relation. The "schema" in the above quote seems
to correspond to more than one relations. Does "schema" in the
above quote mean the same as the schema of a relation?
What is the relation between catalogs and databases? Is the relation
between catalogs and databases one-to-one?
What do the catalogs and schemas look like in mysql, postgresql, or
SQL Server?
Thanks.
Your first sentence in # 1 makes no sense.
A table/relation like “person” has attributes/columns like “name”, “phone”, and “email”.
Tables are grouped together in a namespace known as a schema. So a schema such as “warehouse” can have a table named “person” while another schema such as “sales” can also have a table coincidentally named “person”. Each catalog has one or more schema, each schema carrying a name such as “warehouse” and “sales” seen here.
A schema commonly acts a security boundary, besides being a namespace. As far as I know, that is an implementation detail, not required by the SQL standard.
The word “schema” is also commonly used in a different, more casual and general way, to describe the tables & columns design choices made to fit the needs of an application. See first comment by IMSoP below. A schema in the casual sense might involve any number of catalogs, schemas, tables, and columns in the formal SQL Standard sense.
As for # 2, your quotation explains that. “Catalog” and “database” are synonyms. The word “catalog” is used formally by the SQL standard.
For # 3, advanced databases striving to implement the SQL standard typically support all levels defined by the standard: cluster > catalog > schema > table. This includes both Postgres and Microsoft SQL Server.
H2 Database Engine supports separate databases, each being a catalog with schemas, but no cluster grouping the catalogs/databases together.
MySQL is more limited and does not support the full hierarchy, from what I can tell in my limited searching of MySQL documentation.
For more info, see this related Question: What's the difference between a catalog and a schema in a relational database?
I know that the data in SQL Server is stored in Data pages, But I don't know where the table structure is stored. I came across a statement about TRUNCATE as
"TRUNCATE removes the data by deallocating the data pages.TRUNCATE removes all rows from a table, but the table structure and columns remains"
This made me realize that, table structure, column information is stored outside pages(or Data pages in particular). SO, How/Where is table structure(not data) is stored in SQL server ?
Thank You.
You can access SQL server metadata on INFORMATION_SCHEMA. Following find the most useful views and its content:
INFORMATION_SCHEMA.TABLES: Contains information about the schemas, tables and views in the server.
INFORMATION_SCHEMA.COLUMNS: Full information about the table columns like data type, if it's nullable...
INFORMATION_SCHEMA.VIEWS: Containing information about the views and the code for creating them again.
INFORMATION_SCHEMA.KEY_COLUMN_USAGE: Information about foreign keys, unique keys, primary keys...
To use them, simply query them as they are data views: SELECT * FROM INFORMATION_SCHEMA.TABLES
For a full reference go to MSDN: https://msdn.microsoft.com/en-us/library/ms186778.aspx
There are system tables that store all of the metadata about the database. These tables are not directly queryable (except when using the DAC) but there are numerous views and functions built atop these tables. These are referred to as the Catalog Views.
So, for instance, there is the sys.columns view which describes each column in the database. It's a view built atop the syscolpars table, which is one of the system tables mentioned above that you cannot directly query.
There are also the INFORMATION_SCHEMA views which hespi mentions. These are meant to be a "standard" way of accessing metadata supported by all SQL database systems. Unfortunately, support for them is not 100%, and because they're meant to be cross-platform, they do not tend to reveal advanced features that are product specific.
A SQL Server Database consists of 2 Files (usually):
Master Data File (*.mdf)
Transaction Log File (*.ldf)
)The Master Data File contains: Schema and Data Information
)The Transaction Log Files contains Log Information for Actions in your DB
If you run select * from sys.database_files in your DB it will show you the filenames, location, size, etc..
I've recently had to do some work on an Oracle database. I come from a MS SQL background. I am still trying to get my head around some basic definitions in Oracle
Schema - to me this just meant the structure of the database. Which includes the structure of the tables, indexes and any constrains. This does NOT include any data that is stored in the tables. A database would only contain one Schema and one set of data.
But in Oracle it seems like a Schema is defined as the structure and the data. And a database can hold many Schemas.
Is that accurate?
Regardless of the database engine, it isn't uncommon to talk about your data model as your "schema". That's not necessarily how any relational database engine defines the term but it may be perfectly clear from the context that you're talking only about the definitions of objects and not the actual data.
In both SQL Server and Oracle, a "schema" is a way of collecting together a bunch of related objects, code, and data. If you define a schema in SQL Server and create a table foo in that schema along with a usp_setFoo procedure, the data that is in foo would be part of that schema. In the same way, an Oracle schema would generally involve table and index definitions, data, code, etc.
Technically, in Oracle, a schema is defined as the set of objects owned by a particular user. Practically, an Oracle schema is generally roughly analogous to a SQL Server "database". Oracle normally has two levels of object naming (schema.object) rather than three levels in SQL Server (database.schema.object). If you're using the enterprise edition of Oracle 12.1 with pluggable databases, that changes things a bit and an Oracle pluggable database can be similar to a SQL Server database.
I was wondering if there was a way (relatively simple I hope) to get information about the table and its attributes and realtionships?
Clarification: I want to grab all tables in the database and get the meta-model for the whole database, tables, column data, indicies, unique constraints, relationships between tables etc.
The system has a data dictionary in sys.tables, sys.columns, sys.indexes and various other tables. You can query these tables to get metadata about the database structure. This posting has a script I wrote a few years ago to reverse engineer a database schema. If you take a look at it you can see some examples of how to use the system data dictionary tables.
there are a whole bunch of system views in the information_schema schema in sql server 2005+. is there anything in particular you're wanting?
some of those views include:
check_contraints,
columns,
tables,
views
Try sp_help <tablename>. This will show you foreign key refrences and data about the columns, etc - that is, if you are interested in a specific table, as your question seemed to indicate.
If using .NET code is an option SMO is the best way to do it.
It abstracts away all these system views and tables hiding them behind nice and easy to use classes and collections.
http://msdn.microsoft.com/en-us/library/ms162169.aspx
This is the same infrastructure SQL Server Management Studio uses itself. It even supports scripting.
Abstraction comes at a cost though so you need maximum performance you'd still have to use system views and custom SQL.
You can try to use this library db-meta
I was thinking of putting staging tables and stored procedures that update those tables into their own schema. Such that when importing data from SomeTable to the datawarehouse, I would run a Initial.StageSomeTable procedure which would insert the data into the Initial.SomeTable table. This way all the procs and tables dealing with the Initial staging are grouped together. Then I'd have a Validation schema for that stage of the ETL, etc.
This seems cleaner than trying to uniquely name all these very similar tables, since each table will have multiple instances of itself throughout the staging process.
Question: Is using a user schema to group tables/procs/views together an appropriate use of user schemas in MS SQL Server? Or are user schemas supposed to be used for security, such as grouping permissions together for objects?
This is actually a recommended practice. Take a look at the Microsoft Business Intelligence ETL Design Practices from the Project Real. You will find (download doc from the first link) that they use quite a few schemata to group and identify objects in the warehouse.
In addition to dbo and etl, they also use admin, audit, part, olap and a few more.
I think it's appropriate enough, it doesn't really matter, you could use another database if you liked which is actually what we do.
I'm not sure why you would want a validation schema though, what are you going to do there?
Both the reasons you list (purpose/intent, security) are valid reasons to use schemas. Once you start using them, you should always specify schema when referencing an object (although I'm lazy and never specify dbo).
One trick we use is to have the same-named table in each of several schemas, combined with table partitioning (available in SQL 2005 and up). Load the data in first schema, then when it's validated "swap" the partition into dbo--after swapping the dbo partition into a "dumpster" schema copy of the table. Net Production downtime is measured in seconds, and it's all carefully wrapped in a declared transaction.