How to run tests

Comparison test notice

First of all you need to realize that this test is comparison test, so you need to run at least 2 different versions to compare their performance. If you wish to help us in testing you need to run tests at least with 3 servers (Firebird 1.5, Firebird 2.0 and InterBase 7.5.1). Of course, if you are aware with execution plans magic you can run test for single server version to see what query plans generated.

Step 1 – Install server

Install server you intend to test. To avoid changing paths in toolkits we recommend to install servers into the following locations (if you install server into another location you need to change paths in *.cmd files):

Firebird 1.5 – C:\Firebird15
Firebird 2.0 - C:\Firebird20
InterBase 7.5.1 - C:\InterBase751
Vulcan – you just need to setup environment variable VULCAN which point to Vulcan installation

Step 2 – download and unpack toolkit

After installation you need to download corresponded toolkit.

TPC-R for Firebird 1.5
TPC-R for Firebird 2.0
TPC-R for Vulcan
TPC-R for InterBase 7.5.1

Unzip it to C:\TPC folder. Of course you may choose other location but in this case you need to change paths in all *.sql scripts according to the new location of your database.

Step 3 – create database

Proceed to create directory and run script CREATEDB_FILLDB_IDX.cmd. This script creates database using isql.exe and script tpc_db.sql, populates this database with dbgen.exe tool and create indices from tpc_idx.sql. Database will be created by default in C:\TPC\tpcr.gdb (be careful with gdb extension problem on Windows XP!)

Step 4 – run test

Run script RUN_TEST.cmd from main directory of test. Results will be placed into file log.txt

Step 4 – send log

Please send log.txt (please zip it!) to test@ibdeveloper.com with short description:
hardware (CPU, RAM, HDD/SCSI/etc)
exact server version
database size (it should be ~1.5Gb)
information about you or your company
explicit permission to publish test results from you (if you wish to publish them)

Step 5

Uninstall server, delete database C:\TPC\tpcr.gdb and repeat steps 1-4 for next version you’d like to test.

Please feel free to contact us regarding test execution by test@ibdeveloper.com