Market Data Importing
Start > Programs > Pracplay Glean > Glean, then views > Test > Data Download


Quick Start
Import Tick, Bar or Daily Data for any of Equities, Futures, FX and Options.

Already have data?
    1. Open Glean (start > programs > pracplay glean > glean).
    2. In Glean toolbar, choose views > Test > Download Data.
    3. Then click the Import Tik Data files icon on the toolbar.  (see screenshot above)
    4. Then choose the appropriate converter.   
    5. You may need to try several converters before finding one that works best.
    6. Source isn't listed or doesn't match? You can do customized point-and-click imports using the Generic CSV guide.
    7. Questions or Problems?   Ctrl+H us in Glean.
Need data?  

Converter-Specific Guides



Generic CSV

 

see this post for an example of GenericCSV importing

Requirements :
  • CSV files must have a header to be used with this converter.
  • Each CSV can only have one header
  • the entire CSV must be consistent with it's header (eg no missing columns in row 10 thousand)
  • if importing many csvs at same time, allmust be of the same type and have same header.
Converting Steps

  1. start tikconverter from glean (views > test > data download > tikconverter button)
  2. choose generic csv, click convert
  3. select your CSV file
  4. genericcsv will try to autodetect your type of import (bar, tick level1, ticklevel2, etc). change 'type' if this is incorrect
  5. your source CSV columns are on the left
  6. Destination data fields (for the selected type) are on the right
  7. align the left to match up with the destination fields you wish to import to
  8. a sample import (if successful) will appear on far right as you update your alignments
  9. you can duplicate columns if your data a single format rather than two that Glean expects (eg to match one date field to date and time in the destination set)
  10. you can add a column of a fixed amount if your data is lacking a required field in destination set (eg trade size, bid size, etc)

When ready to perform the import, click import.

Daily Yahoo / Google Data


  1. start tik converter
  2. select data source (google/yahoo)
  3. click convert
  4. enter list of symbols (you can obtain symbol lists for various exchanges from eoddata)
  5. click ok, your symbols will be converted
  6. if you prefer, click view data to access your raw tikfiles

tradestation import (CSV)

  1. your data must be in a specific format to use this converter
  2. if you data is not in this format, you can use excel or google spreadsheets to move the columns and change formats as needed.
  3. export tradestation data as CSV with exactly the following columns in this order
    • DATE = 0
    • TIME = 1
    • OPEN = 2
    • HIGH = 3
    • LOW = 4
    • CLOSE = 5
    • UP = 6
    • DOWN = 7
  4. use the included TikConverter tool and select TradeStation as the source

Cqg purchase

  1. purchase T&S data online from CQG for $6 per symbol month
  2. Make sure to purchase the comma-sepertaed format. Other defaults are fine.
  3. use the included TikConverter tool to convert your files.

eSignal download

  1. in esignal, goto Tools->EFS->Download tick data
  2. enter a symbol, date and number of days to obtain
  3. click save, save somewhere
  4. Use TikConverter in Glean data download view to convert this data, choose eSignal as source format.

IQfeed Importing via Qcollector


Iqfeed data can be imported in several ways.

To import via qcollector :

  • install DTN iqfeed (free trial is available)
  • install qcollector for dtn (trial is available)
  • start qcollector
  • with any ASCII portfolio open, from file menu, choose :
    • add tick data (to add tick by tick data)
    • add bar series (to add bar data)
  • choose ok
  • select symbol for the data
  • select desired dates
  • if bar data, select desired bar interval (minute/day/etc)
  • choose ok, the symbol should appear in your portfolio
  • once you have added all your symbols, right click and choose collect all
  • then start TikConverter (views > designs + data > import tik data) and import the data you just collected using qcollector as the source.

Iqfeed trial is limited to 7 days. Qcollector trial time is unlimited, but restricted to following symbols :

    INTU  (Intuit Inc.)
    USB  
(U.S. Bancorp)
    S  
(Sprint Nextel Corp.)
    ADSK  
(Autodesk Inc.)
    MET  
(MetLife Inc.)
    GT  
(Goodyear Tire & Rubber)

   
In addition, data for four non-stock ticker symbols can be collected:

    FVX
.XO  (5 Yr Treasury Yield index)
    BUSDJPY  
(Forex: Barclays Usd Jpy Spot)
    TRAN
.X  (Dow Jones 20 Transports)
   
@EMD#  (E-mini S&p Midcap 400 continuous futures contract)

Unknown source

  • put your data into same format as one of the existing import types
  • use tik converter to import it
  • if you have bid or ask data, you will want to copy the esignal format (see below)
  • if you only have trade data you can also use esignal or cqg
  • if you have OHLC bar data, most of the formats support this
  • you can use excel to change the order of columns in your data, then save as a csv file to prepare for import
  • See examples below to get ideas

Importing daily data

  • Google
  • Yahoo
  • Euronext

Use TikConverter, select appropriate source and symbols.

Other Supported Data Sources 

  • tradestation
  • esignal
  • cqg
  • TradingPhysics
  • multicharts
  • q collector
  • euronext
  • thompson Reuters
  • trueFX
  • more

If you have problems, try converting your data to CSV and using GenericCSV.

TikConverter Source Column Expectations

You're trying to match expectations TikConverter has for a specific source with however you can export data.   To this end we try to provide a comprehensive list of what TikConverter expects for each source, so even if your source is not listed (or it's different), you can hopefully find a source that matches (or use GenericCSV to make it match).   

You would want to export to match a particular source, or choose an import source that has the same columns and in the same order as your export data. Using excel to add extra (meaningless) columns to match is acceptable.

Here is what tikconverter import sources expect.

export and import matching example

you can open your exported files in notepad, excel or google spreadsheets to see what order of columns they use

good import1 :

COLUMN 1 EXPORT FILE / COLUMN 1 IMPORT FILE
DATE
/ DATE
TIME
/ TIME
PRICE
/ PRICE
SIZE
/ SIZE

bad import1 :

COLUMN 1 EXPORT FILE / COLUMN 1 IMPORT FILE
DATE
/ TIME
TIME
/ DATE
SYMBOL
/ PRICE
SIZE
/ SIZE
PRICE
/ ?

to fix this bad example you'd need to open the file in excel (or google spreadsheets), swap the date and time columns to match, remove the symbol column in the exported file, and swap size and price.

good import2 (probably) :

COLUMN 1 EXPORT FILE / COLUMN 1 IMPORT FILE
DATE
/ DATE
DATE
/ TIME
OPEN
/ OPEN
HIGH
/ HIGH
LOW
/ LOW
CLOSE
/ CLOSE
VOL
/ VOLUME

again just matching up columns. date and time usually can be treated interchangably by the importer (if both time and date are included in same field). in this case the user probably added a second date column that was the exact copy of their first date column, just to get the columns matching.

bad import2 :

COLUMN 1 EXPORT FILE / COLUMN 1 IMPORT FILE
DATE
/ DATE
TIME
/ TIME
SYMBOL
/ SYM
LAST
/ OPEN
SIZE
/ HIGH
BID
/ LOW
BIDSIZE
/ CLOSE
ASK
/ VOL
ASK SIZE
/ ?

this example is bad because the fields/columns don't match, it appears that our export file is level1 data (bid/ask/trade), where the import source selected was bar data (open /high /low/close). To fix this, just choose an import source that matches the export data. If you can't find one that matches exactly, choose the closest and then modify the fields using GenericCSV converter or using excel/google spreadsheets for the other converters.

What follows are the import columns each source expects when reading your export file, as well as the order of those columns.

If you have problems using TikConverter Ctrl+H us in glean.

qcollector columns

        DATE = 0;
        TIME
= 1;
        TRADE
= 2;
        SIZE
= 3;
        EXCH
= 4;
        BID
= 5;
        BIDEX
= 6;
        BIDSIZE
= 7;
        ASK
= 8;
        ASKEX
= 9;
        ASKSIZE
= 10;

supercharts columns

        SYMBOL = 0;
        DATETIME
= 1;
        OPEN
= 2;
        HIGH
= 3;
        LOW
= 4;
        CLOSE
= 5;
        VOL
= 6;

multicharts columns

  • fields of multicharts data - exported from either one tick chart, or as one tick from multicharts quotemanager.
  • tick data in input file should be in the format: date = DD/MM/YYYY, time = HH:MM:SS,Price,Volume
    DATE = 0;
    TIME
= 1;
    PRICE
= 2;
    VOLUME
= 3;

pinnacle columns

    DATE = 0;
    OPEN
= 1;
    HIGH
= 2;
    LOW
= 3;
    CLOSE
= 4;
    VOLUME
= 5;
    OPENINTREST
= 6;

thompson reuters columns

        SYM = 0;
        DATE
= 1;
        TIME
= 2;
        TYPE
= 3;
        PRICE
= 4;
        VOLUME
= 5;
        BID
= 6;
        BIDSIZE
= 7;
        ASK
= 8;
        ASKSIZE
= 9;

example row
in file :

GOOG
,20-APR-2011,13:05:18.881,Quote,,,525.49,10,0,0

drolles columns

        DATE = 0;
        TIME
= 1;
        OPEN
= 2;
        HIGH
= 3;
        LOW
= 4;
        CLOSE
= 5;
        VOL
= 6;

cqg data factory columns

        

        SYM
= 0;
        DATE
= 1;
        TIME
= 3;
        PRICE
= 4;
        TYPE
= 5;

you may want to see https://www.cqgdatafactory.com/docs/CQGDataFactory_FAQ.pdf for more information here.

csv_dtohlcv

    DATE = 0;
    TIME
= 1;
    OPEN
= 2;
    HIGH
= 3;
    LOW
= 4;
    CLOSE
= 5;
    VOLUME
= 6;

tradestation

    DATE = 0;
    TIME
= 1;
    OPEN
= 2;
    HIGH
= 3;
    LOW
= 4;
    CLOSE
= 5;
    UP
= 6;
    DOWN
= 7;

trading physics

        TIME = 0;
        TYPE
= 1;
        SHARES
= 2;
        PRICE
= 3;
        MPID
= 4;

true fx

        SYM = 0;
        DATETIME
= 1;
        BID
= 2;
        ASK
= 3;

generic csv

generic csv supports near-unlimited number of different formats and orders, which you can customize using the graphical interface.

the main goal is the same with any source, make sure export columns match the columns and order that tikconverter expects.

more information on generic csv source is above

euro next google and yahoo

no file export/importing is required here

date time and price formats

Date, time and price formats... most date and time orderings should work. If you have problems and a source includes recommendations you should try to match them.

international users and formats

If you are overseas and having problems using the defaults, try changing your country code to US English in control panel >> region and language, and then try your export/import again (or just make sure you use '.' for decimals and remove any other commas).




Sample Data

  1. 5 equity tick files are included with install, about 1,000 more are downloaded post install.
  2. locate at start > programs > glean > data > view tickdata

Recording Live Data

ASP and Glean record ticks automatically by default for any symbol you trade.