SQL Server: the data dictionary


Distribution of catalog information:
  • What is stored in the master database? What is kept in the user databases?
  • Base metadata tables are stored in the resource database. Metadata views are available in the master database (system-wide information) and in each user database (db specific and some system-wide information).
  • Metadata information can be obtained through catalog views, information schema views, system stored procedures and functions, as well as OLE DB schema rowsets and ODBC catalog funtions.

Obtaining metadata information can be done by using either:
  • (a) System Stored Procedures (T-SQL)
    • Catalog stored procedures
  • (b) System views:
    • Object Catalog views
    • Dynamic Management Views and Functions
    • Information Schema Views

Catalog Stored Procedures and Object Catalog views

sp_columns, sys.columns, and information_schema.columns
sp_columns
USE AdventureWorks2008r2;
GO
EXEC sp_columns department;
Returns 19 columns.. TABLE_QUALIFIER TABLE_OWNER TABLE_NAME COLUMN_NAME ... TYPE_NAME PRECISION LENGTH -------------------- -------------- --------- ------------ ----------------- ---------- ------ AdventureWorks2008R2 HumanResources Department DepartmentID smallint identity 5 2 AdventureWorks2008R2 HumanResources Department Name Name 50 100 AdventureWorks2008R2 HumanResources Department GroupName Name 50 100 AdventureWorks2008R2 HumanResources Department ModifiedDate datetime 23 16 (or)
USE AdventureWorks2008r2;
GO
EXEC sp_columns department, @column_name= 'name';
TABLE_QUALIFIER TABLE_OWNER TABLE_NAME COLUMN_NAME ... TYPE_NAME PRECISION LENGTH -------------------- -------------- --------- ------------ ----------------- ---------- ------ AdventureWorks2008R2 HumanResources Department Name Name 50 100 (for column information, you may also use:)
EXEC sp_help 'HumanResources.department';

sys.columns
Returns a row for each column of an object that has columns (system, internal and user tables, views, table-valued sql function, inline table-valued sql function, table-valued assembly functions).
SELECT name, system_type_id, is_nullable
    FROM sys.columns
    WHERE OBJECT_NAME(OBJECT_ID)= 'Department';
name system_type_id is_nullable ------------ -------------- ------------ DepartmentID 52 0 Name 231 0 GroupName 231 0 ModifiedDate 61 0
SELECT o.name table_name, c.name column_name, t.name data_type,
            t.length length, t.prec precision
    FROM syscolumns c
            INNER JOIN sysobjects o ON o.id = c.id
            LEFT JOIN  systypes t   ON t.xtype = c.TYPE
    WHERE o.TYPE = 'U'
        AND o.name = 'Department'
    ORDER BY o.name, c.name;
table_name column_name data_type length precision ---------- ----------- --------- ------ -------- Department DepartmentID smallint 2 5 Department GroupName NULL NULL NULL Department ModifiedDate datetime 8 23 Department Name NULL NULL NULL

information_schema.columns
Information schema views provide a system table-independent view of the SQL Server metadata.
The idea is that it provides an interface that remains unchanged even after significant changes have been made to the underlying system tables.
USE AdventureWorks2008R2;
GO
SELECT TABLE_NAME, COLUMN_NAME,
            COLUMNPROPERTY(OBJECT_ID(TABLE_SCHEMA + '.' + TABLE_NAME), COLUMN_NAME, 'ColumnID') AS COLUMN_ID
    FROM INFORMATION_SCHEMA.COLUMNS
    WHERE TABLE_NAME = 'Department';
TABLE_NAME COLUMN_NAME COLUMN_ID ---------- ----------- -------- Department DepartmentID 1 Department Name 2 Department GroupName 3 Department ModifiedDate 4

sp_column_privileges and information_schema.column_privileges
sp_column_privileges
USE AdventureWorks2008R2;
GO
EXEC sp_column_privileges 'Employee',
    @table_owner='HumanResources',
    @column_name = 'SalariedFlag';
TABLE_QUALIFIER TABLE_OWNER TABLE_NAME COLUMN_NAME GRANTOR GRANTEE PRIVILEGE IS_GRANTABLE -------------------- -------------- ---------- ------------ ------- ------- --------- --------- AdventureWorks2008R2 HumanResources Employee SalariedFlag dbo dbo INSERT YES AdventureWorks2008R2 HumanResources Employee SalariedFlag dbo dbo REFERENCES YES AdventureWorks2008R2 HumanResources Employee SalariedFlag dbo dbo SELECT YES AdventureWorks2008R2 HumanResources Employee SalariedFlag dbo dbo UPDATE YES
information_schema.column_privileges

SELECT table_name, column_name,column_name, privilege_type, is_grantable
    FROM INFORMATION_SCHEMA.COLUMN_PRIVILEGES
    WHERE TABLE_NAME = 'Department';

sp_special_columns
sp_special_columns
Returns the optimal set of columns that uniquely identify a row in the table.
Returns columns automatically updated when any value in the row is updated.
USE AdventureWorks2008R2;
GO
EXEC sp_special_columns 'Department',
    @table_owner='HumanResources';
SCOPE COLUMN_NAME DATA_TYPE TYPE_NAME PRECISION LENGTH SCALE PSEUDO_COLUMN ----- -------------- --------- ----------------- --------- ------ ----- ------------ 1 DepartmentID 5 smallint identity 5 2 0 1

sp_stored_procedures, sp_sproc_columns, sys.procedures and information_schema.routines
sp_stored_procedures
USE AdventureWorks2008R2;
GO
EXEC sp_stored_procedures;
PROCEDURE_QUALIFIER PROCEDURE_OWNER PROCEDURE_NAME ... ------------------- -------------- ----------------- AdventureWorks2008R2 dbo ufnGetAccountingEndDate;0 ... AdventureWorks2008R2 HumanResources uspUpdateEmployeeHireInfo;1 .. AdventureWorks2008R2 sys sp_spaceused;1 (also)
USE AdventureWorks2008R2;
GO
EXECUTE sp_stored_procedures @sp_owner = N'dbo';

sp_sproc_columns
Returns column information for a stored procedure or user-defined function.
USE AdventureWorks2008R2;
GO
EXEC sp_sproc_columns @procedure_name = 'uspLogError';
PROCEDURE_QUALIFIER PROCEDURE_OWNER PROCEDURE_NAME COLUMN_NAME ... TYPE_NAME ... IS_NULLABLE ------------------- --------------- -------------- ----------- --------- ----------- AdventureWorks2008R2 dbo uspLogError;1 @RETURN_VALUE int NO AdventureWorks2008R2 dbo uspLogError;1 @ErrorLogID int YES

sys.procedures
Returning all stored procedures in a given database;
USE AdventureWorks2008R2;
GO
SELECT name procedure_name,
            SCHEMA_NAME(schema_id) AS schema_name,
            type_desc,
            create_date,
            modify_date
    FROM sys.procedures;

information_schema.routines
Returns the stored procedures and functions that can be accessed by the current user in the current database.
SELECT ROUTINE_SCHEMA, ROUTINE_NAME, ROUTINE_TYPE, ROUTINE_BODY, ROUTINE_DEFINITION
    FROM INFORMATION_SCHEMA.routines;

sp_databases, sys.databases
sp_databases
Lists databases in an instance.
USE MASTER;
GO
EXEC sp_databases;
DATABASE_NAME DATABASE_SIZE REMARKS -------------------- ------------ -------- AdventureWorks2008R2 186240 NULL master 5120 NULL model 1792 NULL msdb 20288 NULL MyDB 10240 NULL tempdb 8704 NULL
sys.databases
USE MASTER;
GO
SELECT name, create_date, compatibility_level, user_access_desc,
            state_desc, recovery_model_desc
    FROM sys.databases;
name create_date compat_level user_access_desc state_desc rec_model_desc ------------------- ----------------------- ------------ ---------------- ---------- -------------- master 2003-04-08 09:13:36.390 100 MULTI_USER ONLINE SIMPLE ... model 2003-04-08 09:13:36.390 100 MULTI_USER ONLINE FULL TESTSQL 2009-06-08 10:43:13.347 100 MULTI_USER ONLINE BULK_LOGGED AdventureWorks2008R2 2009-06-05 17:40:24.250 100 MULTI_USER ONLINE SIMPLE
Other database information views: sys.database_files, sys.database_mirroring, sys.database_recovery_status, sys.master_files


sp_statistics

sp_fkeys


sp_pkeys

sp_table_privileges

sp_server_info

sp_tables








SQL Server: Database Snapshots


  • Available only in the Enterprise version.
  • Snapshots provide read-only, static views of a database.
  • Each snapshot capture the state of all committed data at the point of snapshot creation.
  • Snapshots must be on the same server instance as the source database.
  • Users can query the database snapshot

Why use database snapshots
  • "Freeze data at a point-in-time:" Can be used for reporting data in a particular point in time. (i.e. financial quarter)
  • If snapshot available for query in a mirrored server, it deflects read load from the production server.
  • Protect agains administrative errors:
  • Can be created just before a major change to the database. (i.e. bulk update, schema chenage). Reverting here is much faster than restoring a backup.
  • Protect against user errors:
  • In case of user errors, database can be restored to the time of snapshot creation.
  • Manage a test database: guaratee identical data at each round of testing.

Snapshots use:
  • sparce files
  • copy-on-write operations

How snapshots work
  • When you create a snapshot, no data is necessarily copied from the source database.
  • The snapshot may be thought as being the source database itself.
  • Copy from source to the snapshot occurs only as pages are modified in the source database.
  • The snapshot uses one or more sparse files.
  • Only as DMLs are issued, the original pages are copied to the snapshot before being modified, in a copy-on-write operation.
  • Subsequent updates are not copied to the snapshot.
  • Snapshots contain a catalog of changed pages: a bitmap with the list of pages in the source database that have changed since the snapshot creation.
  • When a page is copied from the source to the snapshot sparse file, its corresponding bit in the catalog of changed pages is updated from o to 1.
  • Growth rate:
    • In this way, the size of the snapshot is proportional to the intensity of database writes on different pages.
    • If most of the updates are repeated changes to a small subset of pages, the growth rate of the snapshot sparse files will be slow.
    • On the other hand, if ALL original pages receive at least one update, the snapshot will grow to the size of the source database.
  • Read operations in a snapshot always access the original data pages, either in the source database (those that have not been modified since the snapshot was created) or in the sparse files with original page data (for the pages modified after the snapshot creation).
Creating a snapshot
  • Use the CREATE DATABASE ... AS SNAPSHOT OF statement. It has to include:
  • One entry for each filename in the source database
  • The logical filename has to be equal to the logical filename in the source database
  • The physical filename indicates where the sparse file will be created.
CREATE DATABASE AdventureWorks2008R2_dbss ON
    ( name = adventureworks2008r2_Data,
            filename = 'c:\mssqldrive\e\AdventureWorks2008R2_Data.ss' )
            AS snapshot OF adventureworks2008r2;
GO

Checking snapshot files and file size:
  • During database snapshot creation, sparse files are created by using the file names in the CREATE DATABASE statement.
  • These file names are stored in sys.master_files in the physical_name column.
  • The size-on-disk of the snapshot files is listed in the size column (in 8-kb page units).
SELECT name, physical_name, type_desc, state_desc, size
    FROM sys.master_files
    WHERE DB_NAME(database_id) = 'AdventureWorks2008R2_dbss';
name physical_name type_desc state_desc size ------------------------- ------------------------------------------- --------- ---------- ------- adventureworks2008r2_Data e:\mssqldrive\e\AdventureWorks2008R2_Data.ss ROWS ONLINE 23024 FileStreamDocuments2008R2 c:\Program...\DATA\Documents2008R2 FILESTREAM ONLINE 0

  • You can also use the fn_virtualfilestats system table-valued function.
  • fn_virtualfilestats takes database_id and file_id as parameters.
SELECT database_id, FILE_ID, physical_name
    FROM sys.master_files
    WHERE DB_NAME(database_id) = 'AdventureWorks2008R2_dbss';
database_id file_id physical_name ----------- ------- ------------ 11 1 e:\mssqldrive\e\AdventureWorks2008R2_Data.ss 11 65537 c:\Program...\DATA\Documents2008R2
SELECT FROM fn_virtualfilestats(11,1);
GO
DbId FileId TimeStamp NumberReads BytesRead ... NumberWrites BytesWritten ... BytesOnDisk ...
---- ------ --------- ----------- ---------     ------------ ------------     -----------
11   1      23108022  195         1597440       163          1466368          3407872

Some restrictions:
  • You cannot backup, restore or detach a database snapshot
  • It must be in the same SQL Server instance as the source database
  • Full-text indexes are not supported
  • You cannot drop, detach or restore a source database if it has a snapshot.
  • You cannot perform structural changes (i.e. add/remove filegroup) to the source database

Dropping a snapshot
DROP DATABASE AdventureWorks2008R2_dbss;

Reverting a database from a Snapshot
Restrictions:
  • Only a single snapshot can exist against a source database before it can be reverted
  • Reverting a database rebuilds the transaction log, so no recovery can be performed aftewards
  • Source database and snapshot database must be offline
USE MASTER;
GO
RESTORE DATABASE AdventureWorks2008R2 
 FROM DATABASE_SNAPSHOT = 'AdventureWorks2008R2_dbss';
GO

Managing Snapshot
(a) Check the MyDB database files
(b) Verify rows on emp table
(c) Create snapshot of MyDB database
(d) Check created snapshot files
(e) Check the size on disk of snapshot files
(f) Insert data on emp table in the source database.
(g) Query snapshot. Emp table is empty.
(a) Check the MyDB database files
sp_helpdb myDb; 

name          fileid filename                           filegroup size    usage
------------  ------ ------------------------------------ --------- -----   --------
MyDB_primary  1      c:\Program...\data\MyDB_Prm.mdf      PRIMARY   4096 KB data only
MyDB_archlog1 2      c:\Program...\data\MyDB_archlog1.ldf NULL      1024 KB log only
MyDB_FG1_Dat1 3      c:\Program...\data\MyDB_FG1_1.ndf    MyDB_fg1  1024 KB data only
MyDB_FG1_Dat2 4      c:\Program...\data\MyDB_FG1_2.ndf    MyDB_fg1  1024 KB data only
MyDB_FG1_Dat3 5      c:\Program...\data\MyDB_FG1_3.ndf    MyDB_fg1  1024 KB data only
MyDB_FG1_Dat4 6      c:\Program...\data\MyDB_FG1_4.ndf    MyDB_fg1  1024 KB data only
MyDB_FG2_Dat1 7      c:\Program...\data\MyDB_FG2_1.ndf    MyDB_fg2  1024 KB data only

(b) Verify rows on emp table
USE MyDB;
GO
SELECT * 
    FROM emp;
empid name ----- ---- (c) Create snapshot of MyDB database
CREATE DATABASE MyDB_dbss ON
    ( name = MyDB_primary,
                filename = 'c:\mssqldrive\e\MyDB_Prm.ss' ),
    ( name = MyDB_FG1_Dat1,
                filename = 'c:\mssqldrive\e\MyDB_FG1_1.ss' ),
    ( name = MyDB_FG1_Dat2,
                filename = 'c:\mssqldrive\e\MyDB_FG1_2.ss' ),
    ( name = MyDB_FG1_Dat3,
                filename = 'c:\mssqldrive\e\MyDB_FG1_3.ss' ),
    ( name = MyDB_FG1_Dat4,
                filename = 'c:\mssqldrive\e\MyDB_FG1_4.ss' ),
    ( name = MyDB_FG2_Dat1,
                filename = 'c:\mssqldrive\e\MyDB_FG2_1.ss' )
            AS snapshot OF MyDB;
GO
(d) Check created snapshot files
SELECT name, physical_name, type_desc, state_desc, size
    FROM sys.master_files
    WHERE DB_NAME(database_id) = 'MyDB_dbss';
name physical_name type_desc state_desc size ------------- ----------------------------- --------- --------- ---- MyDB_primary c:\mssqldrive\e\MyDB_Prm.ss ROWS ONLINE 512 MyDB_FG1_Dat1 c:\mssqldrive\e\MyDB_FG1_1.ss ROWS ONLINE 128 MyDB_FG1_Dat2 c:\mssqldrive\e\MyDB_FG1_2.ss ROWS ONLINE 128 MyDB_FG1_Dat3 c:\mssqldrive\e\MyDB_FG1_3.ss ROWS ONLINE 128 MyDB_FG1_Dat4 c:\mssqldrive\e\MyDB_FG1_4.ss ROWS ONLINE 128 MyDB_FG2_Dat1 c:\mssqldrive\e\MyDB_FG2_1.ss ROWS ONLINE 128 (e) Check the size on disk of snapshot files
SELECT database_id, FILE_ID, physical_name
    FROM sys.master_files
    WHERE DB_NAME(database_id) = 'MyDB_dbss';
database_id file_id physical_name ----------- ------- ------------- 12 1 c:\mssqldrive\e\MyDB_Prm.ss 12 3 c:\mssqldrive\e\MyDB_FG1_1.ss 12 4 c:\mssqldrive\e\MyDB_FG1_2.ss 12 5 c:\mssqldrive\e\MyDB_FG1_3.ss 12 6 c:\mssqldrive\e\MyDB_FG1_4.ss 12 7 c:\mssqldrive\e\MyDB_FG2_1.ss
SELECT * 
    FROM fn_virtualfilestats(12,1);
DbId FileId TimeStamp NumberReads BytesRead ... NumberWrites BytesWritten ... BytesOnDisk ... ---- ------ --------- ----------- --------- ------------ ------------ ----------- 12 1 24067974 50 409600 3 24576 131072 (f) Insert data on emp table in the source database.
USE MyDB;
GO
DECLARE @a INT;
SELECT @a = 1;
WHILE (@a < 100) -- Insert 100 new rows
    BEGIN
    INSERT INTO emp
            VALUES (@a, 'John');
    SELECT @a = @a +1;
    END;
GO
SELECT COUNT(*) num_rows FROM emp;
num_rows ------- 99 (g) Query snapshot. Data is not there.
USE MyDB_dbss;
GO
SELECT COUNT(*) num_rows  FROM emp;
num_rows ------- 0

Reverting database MyDB from database snapshot Mydb_dbss
(a) Check num rows in emp, MyDB database.
(a) Emp table is dropped in MyDB
(b) Revert the database from MyDB_dbss
(c) At the end, Emp table will have been recovered, but without the data inserted/updated after the snapshot had been created.
(a) check table emp;
USE MyDB;
SELECT COUNT(*) num_rows 
    FROM emp;
num_rows ------- 99 (b) drop table emp
USE MyDB;
DROP TABLEemp;
(c) Revert the database snapshot. At the end, the emp table will be available, but without the data inserted after the snapshot creation.
USE MASTER;
GO
RESTORE DATABASE mydb 
 FROM database_snapshot = 'mydb_dbss';
USE MyDB;
SELECT COUNT(*) num_rows 
    FROM emp;
num_rows ------- 0





SQL Server: Moving a Database



Three mechanisms for moving a database (within the same server, or between servers):
  1. Backup and Restore
  2. Copy Database Wizard
  3. Detach/Attach

Detaching/Attaching databases
  • Data files and transaction logs can be detached and then reattached to the same or different SQL Server instance.
  • You may transfer databases across 32-bit and 64-bit environments.

Note that:
  • Make sure you have a backup of the database before attaching it to a newer version server.
  • Once you attach a SQL Server 2000 or 2005 database to a 2008 SQL Server, you will not be able to attach it back into previous version servers.
  • After attaching a database to a new server, you will need to add users from the old server, if they are to continue to have access to the database.

You CANNOT detach a dataabse when:
  • The database is replicated AND published. Before detaching you need to disable publishing (run sp_replicationdboption)
  • Database contains a database snapshot.
  • Database is being mirrored.
  • While there are database sessions.
  • Database is suspect. Needs to be put into emergency mode.
  • Database is a system database.

Moving a database:

(a) Detach the database as follows:
    -- (1) Set database in single_user mode
USE MASTER;
ALTER DATABASE AdventureWorks2008R2
    SET single_user;
GO
    --(2) Check access mode 
SELECT name, user_access_desc
    FROM sys.databases
    WHERE name = 'adventureworks2008r2';
    --(3) Detach database. Set option to skip update statistics for true.
    -- By default, the detach operation retains any out-of-date optimization statistics  
    -- If you want updated optimization statistics to be gathered, set the parameter to 'false'
            .
EXEC sp_detach_db 'AdventureWorks2008R2', 'true';

    --(4) check that the database is no longer listed in the catalog
SELECT name     FROM sys.databases;

(b) MOVE/COPY data and log files to the new location
(c) Re-attach the database as follows:
USE MASTER;
GO
EXEC sp_attach_db @dbname = 'AdventureWorks2008R2' ,
    @filename1 = 'c:\mssqldrive\e\AdventureWorks2008R2_Data.mdf',
    @filename2 = 'c:\mssqldrive\e\AdventureWorks2008R2_Log.ldf';
    -- (2) Check that the database is in the catalog and the new file locations:

EXEC ssp_helpdb Adventureworks2008r2; 
name db_size owner dbid created status compatibility_level -------------------- --------- -------------- ---- ---------- ----------------- -------------------- AdventureWorks2008R2 181.88 MB Pholder\sqltest 9 May 10 2010 Status=ONLINE,... 100 name fileid filename filegroup size ... usage ------------------------- ------ ------------------------------------------ --------- -------- ------ AdventureWorks2008R2_Data 1 g:\sqldata\..\AdventureWorks2008R2_Data.mdf PRIMARY 184192 KB ... data only AdventureWorks2008R2_Log 2 g:\sqldata\..\AdventureWorks2008R2_Log.ldf NULL 2048 KB ... log only

Attaching a database
  • Use sp_attach_db only if the database was previously detached with sp_detach_db, or copied.
  • If you need to specify more than 16 files, use instead
    • CREATE DATABASE dbname FOR ATTACH or
    • CREATE DATABASE dbname FOR_ATTACH_REBUILD_LOG
(more information: here)