In the last Forex Trading Diary Entry (#1) I described how to build an automated trading system that hooks into the OANDA forex brokerage API. I also mentioned that the next steps included constructing a portfolio and risk management overlay for all suggested signals generated by the Strategy component. In this entry of the diary I want to discuss my attempt to build a functioning Portfolio component and how far I've currently progressed.
After writing the last entry, I realised that I really wanted a way to be able to backtest forex strategies in much the same manner as I had demonstrated previously with equities via the event-driven backtester. I wanted there to be as minimal a difference between the live trading environment and the backtesting system. Hence I decided that I needed to build a Portfolio component that would reflect (as much as possible) the current state of the trading account as given by OANDA.
The rationale for this is that the "practice" trading account and the local Portfolio components should have similar, if not equal, values for attributes such as the Account Balance, the Unrealised Profit & Loss (P&L), the Realised P&L and any open Positions. If I could achieve this and run some test strategies through it, and if the attributes appeared to be equal across both the local portfolio object and OANDA, then I could have confidence in the capability of the backtester in producing more realistic results as and when strategies were deployed.
I've spent the last couple of days attempting to implement such a Portfolio object and I believe I've nearly succeeded. I'm still seeing some differences between the local portfolio balance and the OANDA account balance after a number of trades have been carried out.
What are the current limitations of this implementation?
- The base currency, and thus exposure, is hardcoded to be GBP (since I'm in London!). I'll be changing this soon to allow any choice of base currency.
- Currently I've only tested this against GBP/USD, since my base currency is GBP. Shortly I'll modify the exposure calculations to allow any currency pair.
- While some unit testing has suggested that the addition and removal of positions and units is working as I expect it to, it has not yet been tested.
- I've only tested it with opening and closing long positions so far, I've not tested short positions. I'll need to write some unit tests to handle the short positions.
One might sensibly ask why I'm posting it if it has all these limitations? The rationale here is that I want individuals of all levels to realise that building algorithmic trading systems is hard work and requires a lot of attention to detail! There is a significant amount of scope for introducing bugs and incorrect behaviour. I want to outline how "real world" systems are built and show you how to test for these errors and correct them.
I'll start by describing how I constructed the current portfolio setup and then integrated it into the practice trading system that we look at in the previous entry. Then I'll present some reasons for what I think the differences may be.
I've provided all of the code "as is" under the warranty that I stated in the previous entry. If you want to have a play with it and attempt to figure out what might be going wrong, it would be great to discuss it within the Disqus comments below!
Creating the Portfolio
In order to generate a Portfolio object it is necessary to discuss how foreign exchange trades are carried out, since they differ quite substantially from equities.
Calculating Pips and Units
In other asset classes, the smallest increment of a change in asset price is known as a "tick". In foreign exchange trading it is known as a "pip" (Price Interest Point). It is the smallest increment in any currency pair and is (usually) 1/100th of a percent, also known as a basis point. Since the majority of major currency pairs are priced to four decimal places, the smallest change occurs on the last decimal point.
In GPB/USD, for example, a movement from 1.5184 to 1.5185 is one pip (4 decimal places) and thus a pip is equal to 0.0001. Any Japanese Yen based currency makes use of two decimal place pips, so a pip would be equal to 0.01.
The question we can now ask is how much in sterling (GBP) is a movement of 20 pips (20 x 0.0001 = 0.002) equivalent to, per some fixed quantity of units of GBP/USD? If we take 2,000 units of the base currency (e.g. £2,000), then we can calculate the P&L in sterling as follows:
Profit (GBP) = Pips x Exposure / GBPUSD = 0.002 x 2,000 / 1.5185 = £2.63
With OANDA we are free to choose the number of units traded (and thus the exposure generated). Since I have a sterling (GBP) based account and I am trading GBP/USD for this example, the exposure will always equal the number of units. This is currently "hardcoded" into the system below. When I create multiple currency pair options, I will modify the exposure calculation to take into account differing base currencies.
Since the value of the profit described above is quite small, and currencies don't fluctuate a great deal (except when they do!), it is usually necessary to introduce leverage into the account. I'll be discussing this in later articles. For now, we won't need to worry about it.
Overview of Backtesting/Trading System
The current system consists of the following components:
- Event - The Event components carry the "messages" (such as ticks, signals and orders) between the Strategy, Portfolio and Execution objects.
- Position - The Position component represents the concept of a Forex "position", that is a "long" or a "short" in a currency pair with an associated quantity of units.
- Portfolio - The Portfolio component contains multiple Position objects, one for each currency pair being traded. It tracks the current P&L of each position, even after subsequent additions and reductions in units.
- Strategy - The Strategy object takes time series information (curreny pair ticks) and then calculates and sends signal events to the Portfolio, which decide how to act upon them.
- Streaming Forex Prices - This component connects to OANDA over a streaming web socket and receives real-time tick-by-tick data (i.e. bid/ask) from any subscribed currency pairs.
- Execution - Execution takes order events and send them to OANDA to be filled.
- Trading Loop - The trading loop wraps all of the above components together and runs two threads: One for the streaming prices and one for the event handler.
To gain more insight into how the system is connected together, it is worth reading the previous entry in the diary.
Python Implementation
We'll now discuss how I implemented the above system in Python.
Position
The first new component is the Position object. It is designed to replicate the behaviour of an open position in the OANDA fxTrade Practice system. The Positions tab in the fxTrade software contains 8 columns:
- Type - Whether the position is "Long" or "Short"
- Market - Which currency pair to trade, e.g. "GBP/USD"
- Units - The number of units of the currency (see above)
- Exposure (BASE) - The exposure in base currency of the position
- Avg. Price - The average price achieved for multiple purchases. If there are $P$ purchases, this is calculated as $\frac{\sum_{p=1}^P c_p u_p }{\sum_{p=1}^{P} u_p}$, where $c_p$ is the cost of purchase $p$ and $u_p$ are the units acquired for purchase $p$.
- Current - The current sell price.
- Profit (BASE) - The current P&L in the base currency of the position.
- Profit (%) - The current percentage P&L of the position.
As you can see in the following code, these attributes have been reflected in the members of the Position class, with the exception of "Type", which I have renamed to "side", since type is a reserved word in Python!
The class has four non-initialisation methods: calculate_pips, calculate_profit_base, calculate_profit_perc and update_position_price.
The first method, calculate_pips, determines the number of pips that have been generated by this position since it was opened (taking into account any new units added to the position). The second method, calculate_profit_base, calculates the current profit (or loss!) on this position. The third method, calculate_profit_perc, determines the percentage profit on this position. Finally, update_position_price updates the previous two values based on current market data.
class Position(object):
def __init__(
self, side, market, units,
exposure, avg_price, cur_price
):
self.side = side
self.market = market
self.units = units
self.exposure = exposure
self.avg_price = avg_price
self.cur_price = cur_price
self.profit_base = self.calculate_profit_base()
self.profit_perc = self.calculate_profit_perc()
def calculate_pips(self):
mult = 1.0
if self.side == "SHORT":
mult = -1.0
return mult * (self.cur_price - self.avg_price)
def calculate_profit_base(self):
pips = self.calculate_pips()
return pips * self.exposure / self.cur_price
def calculate_profit_perc(self):
return self.profit_base / self.exposure * 100.0
def update_position_price(self, cur_price):
self.cur_price = cur_price
self.profit_base = self.calculate_profit_base()
self.profit_perc = self.calculate_profit_perc()
Since a portfolio can contain multiple positions there will be one class instance for each market that is being traded. As I mentioned above I have only written the Portfolio to handle GBP as the base currency and GBP/USD as the trading instrument. In future articles I will extend the Portfolio object to handle multiple base currencies and multiple currency pairs.
Let's now discuss how to setup a basic virtual environment for Python and then how the Portfolio works.
Virtual Environment Symlink
In the following Portfolio object module I have modified how the imports are handled. I've created a virtual environment, whereby I have added a symlink to my qsforex directory. This allows me to reference a nested hierarchy of project files within each Python module. The code to achieve this in Ubuntu looks something like this:
cd /PATH/TO/YOUR/VIRTUALENV/DIRECTORY/lib/python2.7/site-packages/
ln -s /PATH/TO/YOUR/QSFOREX/DIRECTORY/ROOT/ qsforex
Obviously you will need to replace the locations of your virtual environment and your source code location. I store my virtual environments under the home directory in the ~/venv/ dir. I store my projects under the home directory in the ~/sites/ dir.
This allows me to reference, for instance, from qsforex.event.event import OrderEvent from any file within the project.
Portfolio
The __init__ constructor of the Portfolio requires the following arguments:
ticker- This is the streaming forex prices ticker handler. It is used to get the latest bid/ask prices.events- This is the events queue, which the portfolio needs to palce order events into.base- This is the base currency, in my case this is GBP.leverage- This is the leverage factor. Currently it is 1:20.equity- This is the amount of actual equity in the account, which I've defaulted to £100,000.risk_per_trade- This is the percentage of account equity to risk per trade, which I have defaulted to 2%. This means that the trade units will equal 2,000 for an initial account size of £100,000.
Upon initialisation the class calculates the trade_units, which are the maximum amount of units allowed per position, as well as declaring the positions dictionary (each market is a key) that contains all of the open positions within the portfolio:
from copy import deepcopy
from qsforex.event.event import OrderEvent
from qsforex.portfolio.position import Position
class Portfolio(object):
def __init__(
self, ticker, events, base="GBP", leverage=20,
equity=100000.0, risk_per_trade=0.02
):
self.ticker = ticker
self.events = events
self.base = base
self.leverage = leverage
self.equity = equity
self.balance = deepcopy(self.equity)
self.risk_per_trade = risk_per_trade
self.trade_units = self.calc_risk_position_size()
self.positions = {}
At this stage the "risk management" is rather unsophisticated! In the method calc_risk_position_size below we are simply making sure that the exposure of each position does not exceed risk_per_trade% of the current account equity. risk_per_trade defaults to 2% with the keyword argument, although this can obviously be changed. Hence for an account of £ 100,000, the risk per trade will not exceed £ 2,000 per position.
Note that this figure will not dynamically scale with the size of the account balance, it will only use the initial account balance. Later implementations will incorporate more sophisticated risk and position sizing.
# portfolio.py
def calc_risk_position_size(self):
return self.equity * self.risk_per_trade
The next method, add_new_position, takes the parameters necessary to add a new position to the Portfolio. Notably, it takes the add_price and the remove_price. I have not used the bid and ask prices here directly because the addition and removal prices will depend upon whether the side is "long" or "short". Hence we need to correctly specify which price is which in order to obtain a realistic backtest:
# portfolio.py
def add_new_position(
self, side, market, units, exposure,
add_price, remove_price
):
ps = Position(
side, market, units, exposure,
add_price, remove_price
)
self.positions[market] = ps
We also need a method, add_position_units, which allows units to be added to a position once the position has been created. In order to do this we need to calculate the new average price of the purchased units. Recall that this is calculated by the following expression:
Where $P$ is the number of purchases, $c_p$ is the cost of purchase $p$ and $u_p$ are the units purchased with purchase $p$.
Once the new average price is calculated, the units are updated in the position and then the P&L associated with the position is recalculated:
# portfolio.py
def add_position_units(
self, market, units, exposure,
add_price, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
new_total_units = ps.units + units
new_total_cost = ps.avg_price*ps.units + add_price*units
ps.exposure += exposure
ps.avg_price = new_total_cost/new_total_units
ps.units = new_total_units
ps.update_position_price(remove_price)
return True
Similarly, we need a method to remove the units from a position (but not to close it entirely). This is given by remove_position_units. Once the units and exposure have been reduced the P&L is calculated for the removed units and then added (or subtracted!) from the portfolio balance:
# portfolio.py
def remove_position_units(
self, market, units, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
ps.units -= units
exposure = float(units)
ps.exposure -= exposure
ps.update_position_price(remove_price)
pnl = ps.calculate_pips() * exposure / remove_price
self.balance += pnl
return True
We also need a way to fully close a position. This is given by close_position. It is similar to remove_position_units except that the position is deleted from the positions dictionary:
# portfolio.py
def close_position(
self, market, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
ps.update_position_price(remove_price)
pnl = ps.calculate_pips() * ps.exposure / remove_price
self.balance += pnl
del[self.positions[market]]
return True
The bulk of the work for the class is carried out in the execute_signal method. It takes SignalEvent objects created from the Strategy objects and uses these to generate OrderEvent objects to be placed back to the events queue.
The basic logic is as follows:
- If there is no current position for this currency pair, create one.
- If a position already exists, check to see if it is adding or subtracting units.
- If it is adding units, then simply add the correct amount of units.
- If it is not adding units, then check if the new opposing unit reduction closes out the trade, if so, then do so.
- If the reducing units are less than the position units, simply remove that quantity from the position.
- However, if the reducing units exceed the current position, it is necessary to close the current position by the reducing units and then create a new opposing position with the remaining units. I have not tested this extensively as of yet, so there may still be bugs!
The code for execute_signal follows:
# portfolio.py
def execute_signal(self, signal_event):
side = signal_event.side
market = signal_event.instrument
units = int(self.trade_units)
# Check side for correct bid/ask prices
# Note: This only supports going long
add_price = self.ticker.cur_ask
remove_price = self.ticker.cur_bid
exposure = float(units)
# If there is no position, create one
if market not in self.positions:
self.add_new_position(
side, market, units, exposure,
add_price, remove_price
)
order = OrderEvent(market, units, "market", "buy")
self.events.put(order)
# If a position exists add or remove units
else:
ps = self.positions[market]
# Check if the sides equal
if side == ps.side:
# Add to the position
add_position_units(
market, units, exposure,
add_price, remove_price
)
else:
# Check if the units close out the position
if units == ps.units:
# Close the position
self.close_position(market, remove_price)
order = OrderEvent(market, units, "market", "sell")
self.events.put(order)
elif units < ps.units:
# Remove from the position
self.remove_position_units(
market, units, remove_price
)
else: # units > ps.units
# Close the position and add a new one with
# additional units of opposite side
new_units = units - ps.units
self.close_position(market, remove_price)
if side == "buy":
new_side = "sell"
else:
new_side = "sell"
new_exposure = float(units)
self.add_new_position(
new_side, market, new_units,
new_exposure, add_price, remove_price
)
print "Balance: %0.2f" % self.balance
That concludes the code for the Portfolio class. Now we discuss the event handling.
Event
In order for this Portfolio to function with the new means of generating signals and orders it is necessary to modify event.py. In particular I've added the SignalEvent component, which is now generated by the Strategy object, instead of an OrderEvent. It simply states whether to go long or short a particular "instrument", i.e. currency pair. order_type refers to whether the order is a market order or limit order. I've not yet implemented the latter, so this will remain as "market" for now:
class Event(object):
pass
class TickEvent(Event):
def __init__(self, instrument, time, bid, ask):
self.type = 'TICK'
self.instrument = instrument
self.time = time
self.bid = bid
self.ask = ask
class SignalEvent(Event):
def __init__(self, instrument, order_type, side):
self.type = 'SIGNAL'
self.instrument = instrument
self.order_type = order_type
self.side = side
class OrderEvent(Event):
def __init__(self, instrument, units, order_type, side):
self.type = 'ORDER'
self.instrument = instrument
self.units = units
self.order_type = order_type
self.side = side
Strategy
With the SignalEvent object defined, we also need to change how the Strategy class works. In particular, it now needs to generate SignalEvent events instead of OrderEvents.
I've also changed how the "strategy" actually works. Instead of creating random buy or sell signals, it now generates a buy on every 5th tick and then becomes "invested". On the next 5th tick, if it is invested it simply sells out and becomes "uninvested". This process repeats indefinitely:
from qsforex.event.event import SignalEvent
class TestStrategy(object):
def __init__(self, instrument, events):
self.instrument = instrument
self.events = events
self.ticks = 0
self.invested = False
def calculate_signals(self, event):
if event.type == 'TICK':
self.ticks += 1
if self.ticks % 5 == 0:
if self.invested == False:
signal = SignalEvent(self.instrument, "market", "buy")
self.events.put(signal)
self.invested = True
else:
signal = SignalEvent(self.instrument, "market", "sell")
self.events.put(signal)
self.invested = False
StreamingForexPrices
The Portfolio object requires a ticker object that contains the latest bid and ask prices. I've simply modified the StreamingForexPrices in streaming.py to contain two extra members:
..
..
self.cur_bid = None
self.cur_ask = None
..
..
These are set in the stream_to_queue method:
..
..
if msg.has_key("instrument") or msg.has_key("tick"):
print msg
instrument = msg["tick"]["instrument"]
time = msg["tick"]["time"]
bid = msg["tick"]["bid"]
ask = msg["tick"]["ask"]
self.cur_bid = bid
self.cur_ask = ask
tev = TickEvent(instrument, time, bid, ask)
self.events_queue.put(tev)
As with every object here, the full code can be found below, at the end of the diary entry.
Trading
The final set of modifications occur in the trading.py file. Firstly we modify the imports to take into account the new directory structure and the fact that we're now importing a Portfolio object:
from qsforex.execution.execution import Execution
from qsforex.portfolio.portfolio import Portfolio
from qsforex.settings import STREAM_DOMAIN, API_DOMAIN, ACCESS_TOKEN, ACCOUNT_ID
from qsforex.strategy.strategy import TestStrategy
from qsforex.streaming.streaming import StreamingForexPrices
We then modify the events queue handler to direct SignalEvents to the Portfolio instance:
..
..
while True:
try:
event = events.get(False)
except Queue.Empty:
pass
else:
if event is not None:
if event.type == 'TICK':
strategy.calculate_signals(event)
elif event.type == 'SIGNAL':
portfolio.execute_signal(event)
elif event.type == 'ORDER':
execution.execute_order(event)
time.sleep(heartbeat)
..
..
Main Execution Point
Finally we modify the __main__ function to create the Portfolio and adjust the trade_thread to take the Portfolio as an argument:
..
..
# Create the portfolio object that will be used to
# compare the OANDA positions with the local, to
# ensure backtesting integrity.
portfolio = Portfolio(prices, events, equity=100000.0)
# Create two separate threads: One for the trading loop
# and another for the market price streaming class
trade_thread = threading.Thread(
target=trade, args=(
events, strategy, portfolio, execution
)
)
..
..
Environment Variables in Settings
I also mentioned in the previous article that it is not a good idea to store passwords or other authentication information, including API tokens, in a version controlled codebase. Hence I have modified the settings file to look like this:
import os
ENVIRONMENTS = {
"streaming": {
"real": "stream-fxtrade.oanda.com",
"practice": "stream-fxpractice.oanda.com",
"sandbox": "stream-sandbox.oanda.com"
},
"api": {
"real": "api-fxtrade.oanda.com",
"practice": "api-fxpractice.oanda.com",
"sandbox": "api-sandbox.oanda.com"
}
}
DOMAIN = "practice"
STREAM_DOMAIN = ENVIRONMENTS["streaming"][DOMAIN]
API_DOMAIN = ENVIRONMENTS["api"][DOMAIN]
ACCESS_TOKEN = os.environ.get('OANDA_API_ACCESS_TOKEN', None)
ACCOUNT_ID = os.environ.get('OANDA_API_ACCOUNT_ID', None)
Specifically, the following two lines:
ACCESS_TOKEN = os.environ.get('OANDA_API_ACCESS_TOKEN', None)
ACCOUNT_ID = os.environ.get('OANDA_API_ACCOUNT_ID', None)
I've made use of the os library to retrieve two environment variables (ENVVARS). The first is the API access token and the second is the OANDA account ID. These can be stored in a suitable environment file that is loaded on boot-up of the system. In Ubuntu, you can use the hidden .bash_profile file in your home directory. For instance, using your favourite text editor (mine is Emacs), you can type:
emacs ~/.bash_profile
And add the following two lines, making sure to replace the variables with your own account details:
export OANDA_API_ACCESS_TOKEN='1234567890abcdef1234567890abcdef1234567890abcdef'
export OANDA_API_ACCOUNT_ID='12345678'
You may need to make sure the terminal has access to these variables by running the following from the command line:
source ~/.bash_profile
Running the Code
To get the code running you will need to make sure youre virtual environment is set. I carry this out with the following command (you will need to change this for your particular directory):
source ~/venv/qsforex/bin/activate
You will also need to install the requests library once set, if you didn't do so in the previous article:
pip install requests
Finally, you can run the code (making sure to adjust your path to the project source code):
python qsforex/trading/trading.py
At this point, we're now carrying out our practice trading! As I've stated before in the previous entry, it is very easy to lose money with a system like this hooked up to a live trading account! Make sure to view the disclaimer in the post as well as be extremely careful with your own Strategy objects. I highly recommend trying this on the sandbox or practice accounts prior to a live implementation.
However, before you go ahead and implement this with your own strategies, I'd like to discuss where I think some of the differences between the OANDA account balance and my calculated balance are arising from.
Possible Sources of Error
As the implementation of the systems become more complex, there is a greater risk that bugs have been introduced. I have used some unit testing in order to check the Position and Portfolio behaves as I expect, but there are still discrepancies between the local portfolio and the OANDA account balance. Possible reasons for this include:
- Bugs - Bugs can obviously creep in anywhere. The best way to eliminate them is to have a strong specification upfront about what the program should do and create solid unit tests. I've carried this out for some classes, but not all. Further work is required to have all classes unit tested to a good specification.
- Rounding errors - Since I am using floating point variables to store all financial data, there will be errors in rounding. The way around this is to use Python's Decimal type. Later implementations will utilise the Decimal.
- Slippage - Slippage is the difference between the price that the strategy object saw when deciding to buy or sell and the actual price achieved when the broker executes a fill. Given the multi-threaded nature of the program, slippage is extremely likely to be one of the causes of the differences between the local balance and OANDA account balances.
I will be investigating these issues as I continue to work on the forex system. In the next diary entry I will discuss my progress.
What's Next?
In later articles we are going to discuss the following improvements:
- Differing account balances - The first task is to determine why the account balances differ between OANDA and this local implementation. If anybody has any other ideas, please feel free to add them in the comments!
- Real strategies - I've been reading a few papers on how to apply machine learning to forex markets recently. Converting some of these to actual strategies that we can backtest would be interesting (and fun!).
- Multiple currencies - Adding multiple currency pairs and alternative base currencies.
- Transaction costs - Realistic handling of transaction costs, beyond the bid-ask spread. This will include better slippage modelling and market impact.
There are plenty of other improvements to make as well. This project will be continuously improving!
Full Code
As I mentioned above in order to get this working you will need to create a new virtual environment and symlink it to a directory where the code will live. I have called this directory qsforex. I've referenced it as such below.
Edit: In order for this to work it is necessary to create a file in every directory and subdirectory of the code called __init__.py. I forgot to mention this in the original article. You can carry this out in Mac/Linux by typing touch __init__.py in each of the directories. This will stop ImportErrors from occuring.
Also, remember that this code is a work in progress! I will definitely be making changes in the next week or so and I will update the code to reflect that. Please make sure you test all of this out on your own systems and are happy before applying it to a live trading account.
qsforex/settings.py
import os
ENVIRONMENTS = {
"streaming": {
"real": "stream-fxtrade.oanda.com",
"practice": "stream-fxpractice.oanda.com",
"sandbox": "stream-sandbox.oanda.com"
},
"api": {
"real": "api-fxtrade.oanda.com",
"practice": "api-fxpractice.oanda.com",
"sandbox": "api-sandbox.oanda.com"
}
}
DOMAIN = "practice"
STREAM_DOMAIN = ENVIRONMENTS["streaming"][DOMAIN]
API_DOMAIN = ENVIRONMENTS["api"][DOMAIN]
ACCESS_TOKEN = os.environ.get('OANDA_API_ACCESS_TOKEN', None)
ACCOUNT_ID = os.environ.get('OANDA_API_ACCOUNT_ID', None)
qsforex/event/event.py
class Event(object):
pass
class TickEvent(Event):
def __init__(self, instrument, time, bid, ask):
self.type = 'TICK'
self.instrument = instrument
self.time = time
self.bid = bid
self.ask = ask
class SignalEvent(Event):
def __init__(self, instrument, order_type, side):
self.type = 'SIGNAL'
self.instrument = instrument
self.order_type = order_type
self.side = side
class OrderEvent(Event):
def __init__(self, instrument, units, order_type, side):
self.type = 'ORDER'
self.instrument = instrument
self.units = units
self.order_type = order_type
self.side = side
qsforex/strategy/strategy.py
from qsforex.event.event import SignalEvent
class TestStrategy(object):
def __init__(self, instrument, events):
self.instrument = instrument
self.events = events
self.ticks = 0
self.invested = False
def calculate_signals(self, event):
if event.type == 'TICK':
self.ticks += 1
if self.ticks % 5 == 0:
if self.invested == False:
signal = SignalEvent(self.instrument, "market", "buy")
self.events.put(signal)
self.invested = True
else:
signal = SignalEvent(self.instrument, "market", "sell")
self.events.put(signal)
self.invested = False
qsforex/streaming/streaming.py
import requests
import json
from qsforex.event.event import TickEvent
class StreamingForexPrices(object):
def __init__(
self, domain, access_token,
account_id, instruments, events_queue
):
self.domain = domain
self.access_token = access_token
self.account_id = account_id
self.instruments = instruments
self.events_queue = events_queue
self.cur_bid = None
self.cur_ask = None
def connect_to_stream(self):
try:
s = requests.Session()
url = "https://" + self.domain + "/v1/prices"
headers = {'Authorization' : 'Bearer ' + self.access_token}
params = {'instruments' : self.instruments, 'accountId' : self.account_id}
req = requests.Request('GET', url, headers=headers, params=params)
pre = req.prepare()
resp = s.send(pre, stream=True, verify=False)
return resp
except Exception as e:
s.close()
print "Caught exception when connecting to stream\n" + str(e)
def stream_to_queue(self):
response = self.connect_to_stream()
if response.status_code != 200:
return
for line in response.iter_lines(1):
if line:
try:
msg = json.loads(line)
except Exception as e:
print "Caught exception when converting message into json\n" + str(e)
return
if msg.has_key("instrument") or msg.has_key("tick"):
print msg
instrument = msg["tick"]["instrument"]
time = msg["tick"]["time"]
bid = msg["tick"]["bid"]
ask = msg["tick"]["ask"]
self.cur_bid = bid
self.cur_ask = ask
tev = TickEvent(instrument, time, bid, ask)
self.events_queue.put(tev)
qsforex/portfolio/position.py
class Position(object):
def __init__(
self, side, market, units,
exposure, avg_price, cur_price
):
self.side = side
self.market = market
self.units = units
self.exposure = exposure
self.avg_price = avg_price
self.cur_price = cur_price
self.profit_base = self.calculate_profit_base()
self.profit_perc = self.calculate_profit_perc()
def calculate_pips(self):
mult = 1.0
if self.side == "SHORT":
mult = -1.0
return mult * (self.cur_price - self.avg_price)
def calculate_profit_base(self):
pips = self.calculate_pips()
return pips * self.exposure / self.cur_price
def calculate_profit_perc(self):
return self.profit_base / self.exposure * 100.0
def update_position_price(self, cur_price):
self.cur_price = cur_price
self.profit_base = self.calculate_profit_base()
self.profit_perc = self.calculate_profit_perc()
qsforex/portfolio/portfolio.py
from copy import deepcopy
from qsforex.event.event import OrderEvent
from qsforex.portfolio.position import Position
class Portfolio(object):
def __init__(
self, ticker, events, base="GBP", leverage=20,
equity=100000.0, risk_per_trade=0.02
):
self.ticker = ticker
self.events = events
self.base = base
self.leverage = leverage
self.equity = equity
self.balance = deepcopy(self.equity)
self.risk_per_trade = risk_per_trade
self.trade_units = self.calc_risk_position_size()
self.positions = {}
def calc_risk_position_size(self):
return self.equity * self.risk_per_trade
def add_new_position(
self, side, market, units, exposure,
add_price, remove_price
):
ps = Position(
side, market, units, exposure,
add_price, remove_price
)
self.positions[market] = ps
def add_position_units(
self, market, units, exposure,
add_price, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
new_total_units = ps.units + units
new_total_cost = ps.avg_price*ps.units + add_price*units
ps.exposure += exposure
ps.avg_price = new_total_cost/new_total_units
ps.units = new_total_units
ps.update_position_price(remove_price)
return True
def remove_position_units(
self, market, units, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
ps.units -= units
exposure = float(units)
ps.exposure -= exposure
ps.update_position_price(remove_price)
pnl = ps.calculate_pips() * exposure / remove_price
self.balance += pnl
return True
def close_position(
self, market, remove_price
):
if market not in self.positions:
return False
else:
ps = self.positions[market]
ps.update_position_price(remove_price)
pnl = ps.calculate_pips() * ps.exposure / remove_price
self.balance += pnl
del[self.positions[market]]
return True
def execute_signal(self, signal_event):
side = signal_event.side
market = signal_event.instrument
units = int(self.trade_units)
# Check side for correct bid/ask prices
add_price = self.ticker.cur_ask
remove_price = self.ticker.cur_bid
exposure = float(units)
# If there is no position, create one
if market not in self.positions:
self.add_new_position(
side, market, units, exposure,
add_price, remove_price
)
order = OrderEvent(market, units, "market", "buy")
self.events.put(order)
# If a position exists add or remove units
else:
ps = self.positions[market]
# Check if the sides equal
if side == ps.side:
# Add to the position
self.add_position_units(
market, units, exposure,
add_price, remove_price
)
else:
# Check if the units close out the position
if units == ps.units:
# Close the position
self.close_position(market, remove_price)
order = OrderEvent(market, units, "market", "sell")
self.events.put(order)
elif units < ps.units:
# Remove from the position
self.remove_position_units(
market, units, remove_price
)
else: # units > ps.units
# Close the position and add a new one with
# additional units of opposite side
new_units = units - ps.units
self.close_position(market, remove_price)
if side == "buy":
new_side = "sell"
else:
new_side = "sell"
new_exposure = float(units)
self.add_new_position(
new_side, market, new_units,
new_exposure, add_price, remove_price
)
print "Balance: %0.2f" % self.balance
qsforex/trading/trading.py
import copy
import Queue
import threading
import time
from qsforex.execution.execution import Execution
from qsforex.portfolio.portfolio import Portfolio
from qsforex.settings import STREAM_DOMAIN, API_DOMAIN, ACCESS_TOKEN, ACCOUNT_ID
from qsforex.strategy.strategy import TestStrategy
from qsforex.streaming.streaming import StreamingForexPrices
def trade(events, strategy, portfolio, execution):
"""
Carries out an infinite while loop that polls the
events queue and directs each event to either the
strategy component of the execution handler. The
loop will then pause for "heartbeat" seconds and
continue.
"""
while True:
try:
event = events.get(False)
except Queue.Empty:
pass
else:
if event is not None:
if event.type == 'TICK':
strategy.calculate_signals(event)
elif event.type == 'SIGNAL':
portfolio.execute_signal(event)
elif event.type == 'ORDER':
execution.execute_order(event)
time.sleep(heartbeat)
if __name__ == "__main__":
heartbeat = 0.5 # Half a second between polling
events = Queue.Queue()
# Trade GBP/USD
instrument = "GBP_USD"
# Create the OANDA market price streaming class
# making sure to provide authentication commands
prices = StreamingForexPrices(
STREAM_DOMAIN, ACCESS_TOKEN, ACCOUNT_ID,
instrument, events
)
# Create the strategy/signal generator, passing the
# instrument and the events queue
strategy = TestStrategy(instrument, events)
# Create the portfolio object that will be used to
# compare the OANDA positions with the local, to
# ensure backtesting integrity.
portfolio = Portfolio(prices, events, equity=100000.0)
# Create the execution handler making sure to
# provide authentication commands
execution = Execution(API_DOMAIN, ACCESS_TOKEN, ACCOUNT_ID)
# Create two separate threads: One for the trading loop
# and another for the market price streaming class
trade_thread = threading.Thread(
target=trade, args=(
events, strategy, portfolio, execution
)
)
price_thread = threading.Thread(target=prices.stream_to_queue, args=[])
# Start both threads
trade_thread.start()
price_thread.start()
qsforex/execution/execution.py
mport httplib
import urllib
class Execution(object):
def __init__(self, domain, access_token, account_id):
self.domain = domain
self.access_token = access_token
self.account_id = account_id
self.conn = self.obtain_connection()
def obtain_connection(self):
return httplib.HTTPSConnection(self.domain)
def execute_order(self, event):
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"Authorization": "Bearer " + self.access_token
}
params = urllib.urlencode({
"instrument" : event.instrument,
"units" : event.units,
"type" : event.order_type,
"side" : event.side
})
self.conn.request(
"POST",
"/v1/accounts/%s/orders" % str(self.account_id),
params, headers
)
response = self.conn.getresponse().read()
print response
