Configuring SQL Server: Data and Log files


  1. Configure data files
  2. Configure log files
  3. Configure filegroups






Files (three types:)
Primary data files: (.mdf) Secondary data files: (.ndf) Log files: (.ldf)
Mandatory. Is the database "starting point".
Holds pointers to other files in the database.
Contain schema (structure) and database properties information
Contains startup information for the database catalog.
May (but should not) also contain objects and user data.
Keep only catalog objects in the primary file: reduce disk contention.
These are all data files, other then the primary.
Optional and user-defined.
Contains objects and user data.
Can (should) be distributed across disks/devices to improve access performance.
A database may have up to 32,766 secondary data files.
AT LEAST one log file is needed for each database.
Used for recovery.
Default setting: logs in the same disk in which datafiles are created.
Should be stored in separate disks, unless the database is read-only (no contention, then).



When does the DB Engine uses file location info from primary file (.mdf)?
  • when attaching a database using CREATE DATABASE stmt with either FOR ATTACH or FOR ATTACH_REBUILD_LOG options.
  • When upgrading from SQL Server 2000 or 7.0
  • When restoring the master database.
Note: SQL Server x Oracle
  • In an Oracle database, the list of datafile locations is maintained in the control file, and the information is used for database startup and recovery.
  • In this sense, the primary data file of a SQL Server database, holding file location info, performs a role similar to that performed by Oracle's control file.

  • The locations of all the files in a database are recorded in the primary file AND in the master database.
  • Most of the time, SQLServer DB Engine uses info from the master database.
  • File Logical name: used to refer to the physical files in all Transact-SQL stmts.
  • File Physical (OS) name:
  • NTFS File system recommended.
  • Each instance has its own default directory.
  • Shared files for ALL INSTANCES are located at:
  • disk:\Program Files\Microsoft SQL Server\100\


Data file pages
  • Pages as numbered sequentially (0,1,..)
  • Each file has a unique File ID number
  • To find a page: File ID + page number
  • First page of file: header page.
  • Other system info pages:
    • database boot page: info about the database attributes
    • AIM (allocation information map

(source: SQl Server books online)

Filegroups
Datafiles can be grouped in filegroups for easier administration and better performance.
Filegroup is a logical structure: database objects can be allocated to filegroups, and can be splitted across various filegroups.

Note: SQL Server x Oracle
  • SQL Server: filegroups ==> Oracle: tablespaces
  • An Oracle database, TABLESPACESare logical structures similar to filegroups.
  • Oracle: You assign database objects to a TABLESPACE.
  • SQL Server: You assign database objects to a FILEGROUP.
  • A TABLESPACE may have multiple datafiles, split across different disks or storage devices, for performance improvement.
  • TABLESPACEScan also be backed up or restored as a unit.
  • One object (i.e. table, index) CAN span multiple TABLESPACES, when it is PARTITIONED.
  • Oracle: Default Tablespace => SQL Server: Default filegroup
  • Handling temporary objects:
    • Oracle: Temporary Tablespace => SQL Server: Tempdb database

Primary filegroups:
  • Contains the primary datafile + any secondary datafile not allocated to a filegroup.
  • All system tables are allocated to the primary filegroup.
  • Primary filegroup equivalent to the Oracle SYSTEM TABLESPACE.

Secondary (user-defined) filegroup:
  • User-defined.
  • A databaes may have up to 32,766 secondary filegroups.
  • Created by using FILEGROUP in a CREATE DATABASE or ALTER DATABASE
  • Log files are never part of a filegroup.
  • Each file belongs to ONLY ONE filegroup.
  • If a filegroup has more than one data file, SQL Server distributes the data across the various files.
  • SQL Server keeps the amount of data in each file proportional to the file size.

Filegroup properties:
  • Can be set or changed from SSMS or using ALTER DATABASE statement
  • Default filegroup: One filegroup is designated the DEFAULT filegroup.
  • Members of the db_owner database role can switch the default filegroup.
  • A filegroup can be made READ-ONLY
  • Tables (and other objects) are stored in filegroups, not in files.

Create database MyDB
(1) Define primary filegroup (logical name, physical name, file size, growth policy)
(2) Define secondary filegroup
USE MASTER;
GO
    -- create db with default data filegroup and one log file.
    -- specify growth increment and max fize for primary data file.
CREATE DATABASE MyDB
            ON PRIMARY
    ( name = 'MyDB_primary',
                filename =
        'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_Prm.mdf',
                size=4mb,
                maxsize=10mb,
                filegrowth=1mb),
            filegroup MyDB_fg1
    ( name = 'MyDB_FG1_Dat1',
                filename =
        'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_FG1_1.ndf',
                size=1mb,
                maxsize=10mb,
                filegrowth=1mb),
    ( name = 'MyDB_FG1_Dat2',
                filename =
        'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_FG1_2.ndf',
                SIZE = 1MB,
                MAXSIZE=10MB,
                FILEGROWTH=1MB)
            LOG ON
    ( name='MyDB_archlog1',
                filename =
        'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_archlog1.ldf',
                size=1mb,
                maxsize=10mb,
                filegrowth=1mb);
GO

    -- NOTE: In a production database, the transaction logs SHOULD be stored on a physical disk OTHER THAN 
    -- the one in which the data files are located.

(3) Alter database define default filegroup.
  -- define the default filegroup
ALTER DATABASE MyDB
            modify filegroup MyDB_FG1 DEFAULT;
GO

(4) Add a new data file to a filegroup
  ALTER DATABASE MyDB
    ADD FILE
      ( name = 'MyDB_FG1_Dat3',
        filename = 
         'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_FG1_3.ndf',
        size=1mb,
        maxsize=10mb,
        filegrowth=1mb)
     TO filegroup MyDB_FG1;    

(5) Create table EMP on MyDB
(6) Check table information in the database
  -- create table in the user-defined filegroup
  -- Table EMP created in the dbo schema.
  USE MyDB;
  CREATE TABLE emp
   ( empid INT PRIMARY KEY,
     name CHAR(20))
  ON MyDB_FG1;
  GO   

  EXEC sp_columns @table_name='emp', @table_owner='dbo'
  GO

(7) Create a new filegroup
  -- Create a new filegroup
  ALTER DATABASE MyDB
    ADD filegroup MyDB_FG2;
    
  ALTER DATABASE MyDB
     ADD FILE  
     ( name = 'MyDB_FG2_Dat1',
       filename = 
         'c:\Program Files\Micros...\MSSQL...\MSSQL\data\MyDB_FG2_1.ndf',
       size=1mb,
       maxsize=10mb,
       filegrowth=1mb)       
    TO filegroup MyDB_FG2;;    

sys.databases
sys.filegroups
sys.database_files
sys.master_files
sys.data_spaces

Now, check the database catalog for information on:
(a) Information on databases: sys.databases
USE MyDB;
GO

SELECT name, create_date, user_access_desc,
       state_desc, recovery_model_desc
FROM sys.databases;
name create_date user_access_desc state_desc recovery_model_desc ------- ---------------------- ---------------- ---------- -------------- master 2003-04-08 09:13:36.390 MULTI_USER ONLINE SIMPLE tempdb 2011-10-06 14:56:52.290 MULTI_USER ONLINE SIMPLE model 2003-04-08 09:13:36.390 MULTI_USER ONLINE FULL msdb 2010-04-02 17:35:08.970 MULTI_USER ONLINE SIMPLE TESTSQL 2011-10-08 10:43:13.347 MULTI_USER ONLINE FULL MyDB 2011-10-08 19:12:38.427 MULTI_USER ONLINE FULL

(b) Information on filegroups: sys.filegroups
USE MyDB;
GO

SELECT * FROM sys.filegroups;
name data_space_id type type_desc is_default filegroup_guid log_filegroup_id is_read_only ---------- ------------- ---- ---------------- ---------- ------------------------------------ ---------------- ------------ PRIMARY 1 FG ROWS_FILEGROUP 0 NULL NULL 0 MyDB_fg1 2 FG ROWS_FILEGROUP 1 803DD155-A6A1-4CB5-B183-3FB16D561B13 NULL 0 MyDB_FG2 3 FG ROWS_FILEGROUP 0 D167E2CC-DBB6-45B2-8197-2FF1EA827940 NULL 0 (3 row(s) affected)

(c) Information on data files: sys.database_files
USE MyDB;
GO

SELECT name, type_desc, physical_name, state_desc, size
FROM sys.database_files;
name type_desc physical_name state_desc size ------------- --------- ------------------------------------------------------------------ ---------- ------ MyDB_primary ROWS c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_Prm.mdf ONLINE 512 MyDB_archlog1 LOG c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_archlog1.ldf ONLINE 128 MyDB_FG1_Dat1 ROWS c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_FG1_1.ndf ONLINE 128 MyDB_FG1_Dat2 ROWS c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_FG1_2.ndf ONLINE 128 MyDB_FG1_Dat3 ROWS c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_FG1_3.ndf ONLINE 128 MyDB_FG2_Dat1 ROWS c:\Program Files\Micros...\MSSQL10...\MSSQL\data\MyDB_FG2_1.ndf ONLINE 128

(d) To list the names of the files of all databases in a server:sys.master_files
SELECT DB_NAME(database_id) AS 'db_name', name AS 'logical filename', 
       physical_name, type_desc
FROM sys.master_files;
db_name logical filename physical_name type_desc ---------- ---------------- ------------------------------------------------------- ---------- master master C:\Progr...\MSSQL10...\MSSQL\DATA\master.mdf ROWS master mastlog C:\Progr...\MSSQL10...\MSSQL\DATA\mastlog.ldf LOG tempdb tempdev C:\Progr...\MSSQL10...\MSSQL\DATA\tempdb.mdf ROWS tempdb templog C:\Progr...\MSSQL10...\MSSQL\DATA\templog.ldf LOG model modeldev C:\Progr...\MSSQL10...\MSSQL\DATA\model.mdf ROWS model modellog C:\Progr...\MSSQL10...\MSSQL\DATA\modellog.ldf LOG msdb MSDBData C:\Progr...\MSSQL10...\MSSQL\DATA\MSDBData.mdf ROWS msdb MSDBLog C:\Progr...\MSSQL10...\MSSQL\DATA\MSDBLog.ldf LOG TESTSQL2008 TESTSQL2008 C:\Progr...\MSSQL10...\MSSQL\DATA\InsideTSQL2008.mdf ROWS TESTSQL2008 TESTSQL2008_log C:\Progr...\MSSQL10...\MSSQL\DATA\InsideTSQL2008_log.LDF LOG MyDB MyDB_primary c:\Progr...\MSSQL10...\MSSQL\data\MyDB_Prm.mdf ROWS MyDB MyDB_archlog1 c:\Progr...\MSSQL10...\MSSQL\data\MyDB_archlog1.ldf LOG MyDB MyDB_FG1_Dat1 c:\Progr...\MSSQL10...\MSSQL\data\MyDB_FG1_1.ndf ROWS MyDB MyDB_FG1_Dat2 c:\Progr...\MSSQL10...\MSSQL\data\MyDB_FG1_2.ndf ROWS MyDB MyDB_FG1_Dat3 c:\Progr...\MSSQL10...\MSSQL\data\MyDB_FG1_3.ndf ROWS

(e) check data spaces:
USE MyDB;
SELECT name, type_desc FROM sys.data_spaces;
name type_desc -------- -------------- PRIMARY ROWS_FILEGROUP MyDB_fg1 ROWS_FILEGROUP MyDB_FG2 ROWS_FILEGROUP

Database files and filegroups
(figure modified from here.)

Oracle: the company and the database. A timeline





Summer 1966
  • Ellison moves to Berkeley, CA. Works weekends as IBM systems programmer

1973
  • Ellison leaves IBM and start working for Ampex (company invented the magnetic tape recorder).
  • Works in the terabit memory system project, code-named Oracle, funded by the CIA, at Ampex.
  • At Ampex, worked in a database management system for the PDP-11 minicomputer. A CODASYL database, modeled after Cullinet's IDMS mainframe database.


1977
Larry Ellison read Don Chamberlain's papers on the System R and SQL language and decided to build a relational database management system for minicomputers.
Ellison (34 y/o), Bob Miner and Ed Oates founded Software Development Laboratories.
In the picture: Ed Oates, Bruce Scott, Bob Miner and Larry Ellison.
Bruce Scott, who would be hired upon the formation of the company, is the “Scott” in scott/tiger (tiger was Bruce’s daughter’s cat).


1978
SDL signed a contract with CIA to continue working on a SQL-based relational database.
Oracle Version 1, written in assembly language, runs on PDP-11 under RSX, in 128K of memory.
Oracle V1 is never officially released.

1979
SDL changes its name to Relational Software Inc.
RSI launches Oracle 2.1. The first commercial relational database using SQL.
Still written in PDP-11 assembly language

1982
RSI becomes Oracle Systems.


1983
  • Oracle launches Oracle V3.
  • Written in C, is the first commercially available portable RDBMS. It run on a range of hardware and operating systems, including mainframes, minicomputers, workstations and PCs.
  • Oracle V3 introduced support for transactions, atomic execution of SQL statements, nonblocking queries (avoided read locks).
  • It was also the first to provide support for symmetric multiprocessing (SMP)

1984
Oracle launches Oracle Version 4.
Version 4 supported read-consistency, and introduced the export/import utility

Check some screen shots and material from Version 4.1:


1985
Launched Oracle Version 5.
Oracle now supported a client-server model and distributed queries.
First support for distributed database


1986
Oracle made its first public offering of stock.
One of the principal reasons for Oracle's success was the 1986 emergence of SQL as the industry's standard language for relational database management systems, which in turn led to increased market acceptance of Oracle's SQL-compatible RDBMS.
Oracle releases SQL*Plus, SQL*Menu, SQL*Graph

In 1986 Oracle expanded its RDBMS product line and debuted another industry first, a distributed DBMS based on the company's SQL*Star software.

racle SQL*Star software was the first commercially available software of its kind and was soon expanded to include dozens of additional computer brands and models.

1988
Oracle Version 6 launched.
Technical advances:
Row-level locking, Hot backup, and first version of PL/SQL.
PL/SQL was not yet stored in the database. It was supported embedded within Forms V3.

1989
Oracle provides database support of online transaction processing (OLTP).

1990
Oracle Applications Release 8 launched (Accounting).

1992
Oracle Version 7 launched.
Version 7 supported declarative referential integrity, stored procedures and triggers.


1996
Oracle launches Release 7.3.
Oracle 7.3 offered the Universal Server.
Provided datatype support for text, video, maps, sound, images datatypes.


1998
Oracle 8 and Oracle Applications 10.7 launched.
Added ability to create and store objects in the database.
Added support for Java. First database to incorporate a native JRE.
First proprietary database available in Linux.





1999
Release of Oracle 8i.
8i incorporated a native Java virtual machine (Oracle JVM).





2000
Oracle E-Business suite 11i launched.

2001
Oracle 9i launched. 400 new features.
Support for Real Application Clusters (RAC) replaced Oracle Parallel Server (OPS).
Added new data warehousing features



2003
Oracle 10g released.
Enabled "grid" computing: computer and software resources provided for applications on an as-needed basis.
Added the ability to provision CPUs and data.
Introduction of self-managing features:
  • Automated Database Diaginostic Monitor (ADDM)
  • Automated Shared Memory Tuning,
  • Automated Storage Management (ASM)
  • Automated Disk based Backup and Recovery













2005
Oracle 10g Release 2 (10gR2) launched.
Oracle acquires PeopleSoft and announces its intention to buy Siebel Systems.



2007
Oracle 11g released.
11g brought many new features, including Oracle Total Recall (Flashback Data Archive), hot patching, automate capture of fault diagnostics and provision of repair advisors and integration with Oracle Metalink.
Peoplesoft Enterprise 9 launched.
Siebel's CRM 8 launched.
Oracle Virtual Machine launched.
Oracle acquires:
  • HYPERION - provider of Enterprise Performance Management (EPM) software. Later couples with Oracles BI tools and analytic applications.
  • AGILE - provider of product lifecycle management (PLM) solutions. (Management of complete product lifecycle, from conception and desing to production, sales and service).
  • TANGOSOL - provider of in-memory data grid software. (Added to the Oracle Fusion Middleware infrastructure, to support SOA and EDA).
  • Patents from APPFORGE - in the area of mobile applications
  • LODESTAR - provider of meter data management and competitive energy operation solutions.



2008
Oracle acquires:
  • BEA Web systems - provider of enterprise application infrastructure solutions (Java app servers, transaction processing monitors, SOA and business process management).
  • CAPTOVATION - provider of document capture solutions.
  • EMPIRIX - acquires the e-TEST suite of products (to add to OEM)

2009
Oracle acquires:
  • Primavera Software - provider of project portfolio management solutions (engineering, construction, public sector, aerospace and defense, utilities, oil and gas, manufacturing and high technology).
  • Haley - provider of policy modeling and automation software for legislative and regulated industries (public sector, financial services and insurance).
  • Skywire Software - provider of insurance and document management business applications. (Expands Oracle Enterprise Content management Suite)
  • AdminServer - provider of insurance policy administration software
  • Relsys - provider of drug safety and risk management solutions.
  • mValent - provider of application configuration management solutions.

2010
Oracle acquires SUN Microsystems.
Other acquisitions:
  • AmberPoint - SOA management provider
  • Silver Creek - data quality products
  • Convergin - developer in Java 2, Service Broker and network integration software
  • Passlogix - Enterprise singe sign-on and network authentication capabilities (Oracle Identity management)
  • Secerno - heterogenous database firewall ( became Oracle db firewall)
  • Phase Forward - provider of applications for healthcare providers and life science companies.



Sources and Further reading:
Oracle Corporate/technical timeline:
Oracle timeline at oracle.com

Oracle database explained
Oracle Timeline at Profit magazine
On Bruce Scott

(6) Control Structures: CONTINUE statement





CONTINUE statement
  • CONTINUE jumps out of the current LOOP interaction and STARTS THE NEXT ONE.
  • Can be used on its own or as part of a CONTINUE WHEN

set serveroutput on 
declare 
 l_number number := 0;
begin
 for i in 1 .. 10 LOOP
                                          -- code before CONTINUE: executed
   dbms_output.put_line('Before Continue: Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
   CONTINUE WHEN MOD(i,2) = 0;
                                          -- If WHEN condition TRUE, code after continue: Jumped.
   dbms_output.put_line('Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
   l_number := l_number +1;
 end loop;
 
 dbms_output.put_line('CONTINUE WHEN:'|| l_number);
end;

Alternativelly:
set serveroutput on 
declare 
 l_number number := 0;
begin
 for i in 1 .. 10 LOOP
   IF mod(i,2) = 0 THEN 
     CONTINUE;
   END IF;
   dbms_output.put_line('Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
   l_number := l_number +1;
 end loop;

  • Before the CONTINUE statement, the same behavior could be implemented with IF, GOTO or EXCEPTIONS..:
-- Implementing the same behavior using Exception:
set serveroutput on 
declare 
 e_continue exception;
 ln number :=0;
begin 
 for i in 1 .. 10 loop
  begin
   if mod(i,2) != 0 then
     raise e_continue;
   end if;
    dbms_output.put_line('Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
    ln := ln + 1;   
  exception
   when e_continue then
     null;
  end;
 end loop;
 dbms_output.put_line('EXCEPTION: ' || ln);

set serveroutput on 
declare 
 ln number :=0;
begin
 ln := 0;
 FOR i IN 1 .. 10 LOOP
    IF MOD(i,2) != 0 THEN
      dbms_output.put_line('Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
      ln := ln + 1;   
    END IF;
  END LOOP;
  dbms_output.put_line('IF       : ' || ln);
end;

set serveroutput on 
declare 
 ln number :=0;
begin
  ln := 0;
  FOR i IN 1 .. 10 LOOP
    IF MOD(i,2) = 0 THEN
      GOTO label_continue;
    END IF;
    dbms_output.put_line('Iteracao # '|| i ||'. MOD('||i||', 2) = '|| mod(i,2));
    ln := ln + 1;   

    << label_continue >>
    NULL;
  END LOOP;
  dbms_output.put_line('GOTO     : ' || ln);
end;