I was trying to analyze the performance test results of a customer facing e-commerce application. Number of concurrent users were close to 700 and the load testing tool (HP performance center) picked these users from a range of 100K records stored in flat files. My objective was to find out the number of orders placed by each of these users and confirm that they are uniformly distributed.
To begin with, I tried opening the flat files (there were many of them, each script had its own flat file for storing user ids) and noted down the range of user ids for comparing the orders. I soon realized that the user ids stored in flat file were not ordered and my range logic is not going to work. That's when I thought about loading these data into a database table so that analysis is much simpler.
I tried creating INSERT scripts for each record using a macro and tried running the script for the first 65 K records. The tool I was using threw out of memory error and I needed a better solution.
Oracle SQL* Loader
I googled a bit and found that Oracle SQL*Loader can do this easily, here is one blog which explains how - http://www.thegeekstuff.com/2012/06/oracle-sqlldr/
Key part of this is the creation of a control file where the input file, target table and other config parameters are specified.
My control file looks like below - save this as sqlldr-tusertest.ctl
Now for loading this using SQL*Loader, execute below command
To begin with, I tried opening the flat files (there were many of them, each script had its own flat file for storing user ids) and noted down the range of user ids for comparing the orders. I soon realized that the user ids stored in flat file were not ordered and my range logic is not going to work. That's when I thought about loading these data into a database table so that analysis is much simpler.
I tried creating INSERT scripts for each record using a macro and tried running the script for the first 65 K records. The tool I was using threw out of memory error and I needed a better solution.
Oracle SQL* Loader
I googled a bit and found that Oracle SQL*Loader can do this easily, here is one blog which explains how - http://www.thegeekstuff.com/2012/06/oracle-sqlldr/
Key part of this is the creation of a control file where the input file, target table and other config parameters are specified.
My control file looks like below - save this as sqlldr-tusertest.ctl
load data
infile '/home/kiran/userids.txt'
into table T_USERTEST
fields terminated by "|"
Now for loading this using SQL*Loader, execute below command
$sqlldr <dbuserid>/<dbpassword>@<dbhost>:<port>/<sid> control=/home/kiran/sqlldr-tusertest.ctl log=/home/kiran/sqlldr.log
No comments:
Post a Comment