Monthly Archives: March 2020

MathWorks Publishes Machine Learning Application Using LOBSTER Data

Posted in LOBSTER Data | Leave a comment

Self Type in Scala

abstract class A (val x: Int){ } trait B { def xPlus1: Int = x +1 } Of course this is silly. Because B does not know x defined in A, Scala compiler will complain, error: not found: value x … Continue reading

Posted in Scala | Leave a comment

Mount AWS S3 Bucket on An EC2 Ubuntu Instance

The current Ubuntu version on AWS EC2 instances is 18.04 LTS. After launch an Ubuntu Instance and connect to it by SSH, one can follow those steps to set up a permanent mount to S3 bucket. Install the packages. $ … Continue reading

Posted in AWS | Leave a comment

Convert A Spark DataFrame with Date Columns to Pandas DataFrame using Apache Arrow

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. After install pyarrow package, we can convert Spark DataFrame … Continue reading

Posted in Python, Spark | Leave a comment

Tips on Linux

How to unzip all 7z files in a directory? One can use the following bash script to unzip all *.7z files in a directory. #!/bin/bash FILES=$1/*.7z shopt -s nullglob for f in $FILES do echo "Processing $f file …" 7za … Continue reading

Posted in Linux | Leave a comment

Evaluate Trading Strategies by Using LOBSTER Data

— A short review of a working paper by Balch (2019) from J.P. Morgan Artificial Intelligence Research and Imperial College London T.H. Balch have published a working paper, “How to Evaluate Trading Strategies: Single Agent Market Replay or … Continue reading

Posted in LOBSTER Data | Leave a comment