Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binancial secs: futes for the peepz! #520

Open
wants to merge 73 commits into
base: storage_middleware_layer
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
bc4ded2
binance: start drafting live order ctl endpoints
Jun 21, 2021
f99e8fe
binance: dynamically choose the rest method
goodboy Jun 9, 2023
d3a5048
Add draft `brokercnf` CLI cmd from @guilledk
goodboy Jun 9, 2023
1d9c195
kraken: tidy up paper mode activation comments
goodboy Jun 9, 2023
f8af13d
binance: add `submit_cancel()` & listen key mgmt
goodboy Jun 9, 2023
e03da40
Add a config get/set API (from @guilledk) ?
goodboy Jun 9, 2023
e85e031
Use new config get/set API in `brokercnf` cmd?
goodboy Jun 9, 2023
ef544ba
Add order status tracking
Jul 1, 2021
eaaf6e4
kraken: fix `trades2pps()` type sig
goodboy Jun 9, 2023
7c00ca0
binance: add deposits/withdrawals API support
Feb 19, 2022
06026ec
Add `binance` section to broker conf template
goodboy Jun 9, 2023
2dc8ee2
Don't bother casting `AggTrade` values for now, just floatify the pri…
goodboy Jun 9, 2023
e035af2
Don't filter out clearing ticks XD
goodboy Jun 9, 2023
26a8638
binance: convert to subpkg module
goodboy Jun 9, 2023
ed0c255
binance: make pkgmod expose endpoints from coming submods
goodboy Jun 9, 2023
ae1c5a0
binance: breakout into `feed` and `broker` mods like other backends
goodboy Jun 9, 2023
dac93dd
Support USD-M futes live feeds and exchange info
goodboy Jun 10, 2023
aa49c38
Add `binance` section to `brokers.toml`
goodboy Jun 12, 2023
8220bd1
Extend `MktPair` doc string to refer to binance pairs
goodboy Jun 12, 2023
d173d37
kraken: raise `SymbolNotFound` on symbology query errors
goodboy Jun 13, 2023
dc23f1c
binance: fix `FutesPair` to have `.filters`
goodboy Jun 13, 2023
e68c55e
Switch `Client.mkt_mode` to 'usd_futes' if 'perp' in fqme
goodboy Jun 13, 2023
4c4787c
Add a "perpetual_future" mkt info type
goodboy Jun 13, 2023
8e03212
Always expand FQMEs with .venue and .expiry values
goodboy Jun 14, 2023
9ff03ba
kraken: add `<pair>.spot.kraken` fqme interpolation
goodboy Jun 14, 2023
3c89295
Rename `.binance.schemas` -> `.venues`
goodboy Jun 14, 2023
f792ecf
binance: use new `open_trade_dialog()` endpoint name B)
goodboy Jun 14, 2023
9972bd3
kraken: use new `open_trade_dialog()` ep name B)
goodboy Jun 14, 2023
0c74a67
Move API urls to `.venues`
goodboy Jun 14, 2023
2ee11f6
binance: facepalm, always lower case venue token..
goodboy Jun 14, 2023
1bb7c9a
Handle pending futes, optional `.filters` add testnet urls
goodboy Jun 17, 2023
c6d1007
Load `Asset`s during echange info queries
goodboy Jun 17, 2023
43494e4
Add note about expecting client side to cache search domain?
goodboy Jun 17, 2023
f36061a
binance: first draft live order ctl support B)
goodboy Jun 17, 2023
98f6d85
Make order request methods be venue aware
goodboy Jun 17, 2023
a9c016b
Use `Client._pairs` cross-venue table for orders
goodboy Jun 17, 2023
249d358
Woops, fix wss_url lookup depending on venue..
goodboy Jun 17, 2023
60b0b72
Split out crypto$ derivs into separate type set
goodboy Jun 17, 2023
45ded4f
binance: order submission "user id" is not the same as their internal…
goodboy Jun 17, 2023
8a06e4d
Wrap dialog tracking in new `OrderDialogs` type, info log all user st…
goodboy Jun 17, 2023
09007cb
Do native symbology lookup in order methods, send user oid in cancel …
goodboy Jun 17, 2023
3f555b2
Fix user event matching
goodboy Jun 17, 2023
4eeb232
kraken: add more type annots in broker codez
goodboy Jun 17, 2023
572badb
Add full real-time position update support B)
goodboy Jun 17, 2023
6eee6ea
binance: add accounts def to `brokers.toml` template
goodboy Jun 17, 2023
dc3ac8d
binance: support order "modifies" B)
goodboy Jun 18, 2023
5c315ba
Support live order loading (with caveats)
goodboy Jun 19, 2023
7f39de5
Factor `OrderDialogs` into `.clearing._util`
goodboy Jun 19, 2023
77db2fa
Support loading quarterly futes existing lives
goodboy Jun 19, 2023
fe902c0
Drop `OrderedDict` usage, not necessary in modern python
goodboy Jun 19, 2023
9970fa8
Drop per-venue request methods from `Client`
goodboy Jun 19, 2023
676b005
Don't allow `Client.api()` testnet queries by default, require explic…
goodboy Jun 19, 2023
e4c1003
Hard code futes venue(s) for now in `brokerd`..
goodboy Jun 19, 2023
5d93017
kraken: use new `OrderDialogs` type, handle `.spot`
goodboy Jun 20, 2023
d82173d
Always use fully expanded FQME throughout `.clearing`
goodboy Jun 20, 2023
a4d16ec
Fix ems tests: add `.spot` venue token to fqme
goodboy Jun 20, 2023
65f2549
binance: more explicit var naming in `OHLC` parse loop
goodboy Jun 20, 2023
d9708e2
kraken: drop `OHLC.ticks` field and just inject to quote before send
goodboy Jun 20, 2023
3fcf44a
Skip marketstore docker tests, we're gonna drop it..
goodboy Jun 20, 2023
f7f7613
kraken: handle `.spot.kraken` new-style FQMEs
goodboy Jun 22, 2023
b1ef549
Move `broker_init()` into `brokers._daemon`
goodboy Jun 23, 2023
3be1d61
ib: expose trade EP as `open_trade_dialog()`
goodboy Jun 24, 2023
efd52e8
kraken: always insert ticks `list`, only append if vlm
goodboy Jun 24, 2023
cbe364c
Add explicit `piker.cli` logger name for `pikerd`
goodboy Jun 24, 2023
032976b
view_mode: add in one missing debug_print block..
goodboy Jun 24, 2023
cf1f4be
Move `.accounting` related config loaders to subpkg
goodboy Jun 25, 2023
2d291bd
ib: expose `.broker.norm_trade_records()` from pkg
goodboy Jun 25, 2023
249b091
binance: better bad account in order request error msg
goodboy Jun 26, 2023
85fa87f
Update the `_emsd_main()` doc task tree layout
goodboy Jun 26, 2023
49e67d5
Always add a paper (account) entry to order mode init
goodboy Jun 26, 2023
cdf9105
Export `Flume` and `Feed` from `piker.data`
goodboy Jun 26, 2023
e7e7919
Ensure paper engine logger is `piker.clearing` instance..
goodboy Jun 26, 2023
4a8eafa
Never key error on bad flow pops..
goodboy Jun 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 25 additions & 7 deletions config/brokers.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
[questrade]
refresh_token = ''
access_token = ''
api_server = 'https://api06.iq.questrade.com/'
expires_in = 1800
token_type = 'Bearer'
expires_at = 1616095326.355846
################
# ---- CEXY ----
################
[binance]
accounts.usdtm = 'futes'
futes.use_testnet = true
futes.api_key = ''
futes.api_secret = ''

accounts.spot = 'spot'
spot.use_testnet = true
spot.api_key = ''
spot.api_secret = ''


[deribit]
Expand All @@ -24,6 +30,18 @@ key_secret = ''
key_passphrase = ''


################
# -- BROKERZ ---
################
[questrade]
refresh_token = ''
access_token = ''
api_server = 'https://api06.iq.questrade.com/'
expires_in = 1800
token_type = 'Bearer'
expires_at = 1616095326.355846


[ib]
hosts = [
'127.0.0.1',
Expand Down
2 changes: 2 additions & 0 deletions piker/accounting/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
MktPair,
Symbol,
unpack_fqme,
_derivs as DerivTypes,
)
from ._allocate import (
mk_allocator,
Expand All @@ -65,6 +66,7 @@
'open_pps',
'open_trade_ledger',
'unpack_fqme',
'DerivTypes',
]


Expand Down
48 changes: 46 additions & 2 deletions piker/accounting/_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,11 @@ def update_from_t(
self,
t: Transaction,
) -> None:
'''
Given an input `Transaction`, cast to `dict` and update
from it's transaction id.

'''
self.data[t.tid] = t.to_dict()

def iter_trans(
Expand Down Expand Up @@ -259,6 +264,45 @@ def dyn_parse_to_dt(
yield tid, data


def load_ledger(
brokername: str,
acctid: str,

) -> tuple[dict, Path]:
'''
Load a ledger (TOML) file from user's config directory:
$CONFIG_DIR/accounting/ledgers/trades_<brokername>_<acctid>.toml

Return its `dict`-content and file path.

'''
import time
try:
import tomllib
except ModuleNotFoundError:
import tomli as tomllib

ldir: Path = config._config_dir / 'accounting' / 'ledgers'
if not ldir.is_dir():
ldir.mkdir()

fname = f'trades_{brokername}_{acctid}.toml'
fpath: Path = ldir / fname

if not fpath.is_file():
log.info(
f'Creating new local trades ledger: {fpath}'
)
fpath.touch()

with fpath.open(mode='rb') as cf:
start = time.time()
ledger_dict = tomllib.load(cf)
log.debug(f'Ledger load took {time.time() - start}s')

return ledger_dict, fpath


@cm
def open_trade_ledger(
broker: str,
Expand All @@ -267,7 +311,7 @@ def open_trade_ledger(
# default is to sort by detected datetime-ish field
tx_sort: Callable = iter_by_dt,

) -> Generator[dict, None, None]:
) -> Generator[TransactionLedger, None, None]:
'''
Indempotently create and read in a trade log file from the
``<configuration_dir>/ledgers/`` directory.
Expand All @@ -277,7 +321,7 @@ def open_trade_ledger(
name as defined in the user's ``brokers.toml`` config.

'''
ledger_dict, fpath = config.load_ledger(broker, account)
ledger_dict, fpath = load_ledger(broker, account)
cpy = ledger_dict.copy()
ledger = TransactionLedger(
ledger_dict=cpy,
Expand Down
41 changes: 40 additions & 1 deletion piker/accounting/_mktinfo.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
from ..data.types import Struct


# TODO: make these literals..
_underlyings: list[str] = [
'stock',
'bond',
Expand All @@ -47,6 +48,10 @@
'commodity',
]

_crypto_derivs: list[str] = [
'perpetual_future',
'crypto_future',
]

_derivs: list[str] = [
'swap',
Expand All @@ -66,6 +71,8 @@
_underlyings
+
_derivs
+
_crypto_derivs
]

# egs. stock, futer, option, bond etc.
Expand Down Expand Up @@ -207,6 +214,33 @@ class MktPair(Struct, frozen=True):
<dst>/<src>.<expiry>.<con_info_1>.<con_info_2>. -> .<venue>.<broker>
^ -- optional tokens ------------------------------- ^


Notes:
------

Some venues provide a different semantic (which we frankly find
confusing and non-general) such as "base" and "quote" asset.
For example this is how `binance` defines the terms:

https://binance-docs.github.io/apidocs/websocket_api/en/#public-api-definitions
https://binance-docs.github.io/apidocs/futures/en/#public-endpoints-info

- *base* asset refers to the asset that is the *quantity* of a symbol.
- *quote* asset refers to the asset that is the *price* of a symbol.

In other words the "quote" asset is the asset that the market
is pricing "buys" *in*, and the *base* asset it the one that the market
allows you to "buy" an *amount of*. Put more simply the *quote*
asset is our "source" asset and the *base* asset is our "destination"
asset.

This defintion can be further understood reading our
`.brokers.binance.api.Pair` type wherein the
`Pair.[quote/base]AssetPrecision` field determines the (transfer)
transaction precision available per asset; i.e. the satoshis
unit in bitcoin for representing the minimum size of a
transaction that can take place on the blockchain.

'''
dst: str | Asset
# "destination asset" (name) used to buy *to*
Expand Down Expand Up @@ -513,10 +547,15 @@ def quantize(
# TODO: BACKWARD COMPAT, TO REMOVE?
@property
def type_key(self) -> str:

# if set explicitly then use it!
if self._atype:
return self._atype

if isinstance(self.dst, Asset):
return str(self.dst.atype)

return self._atype
return 'unknown'

@property
def price_tick_digits(self) -> int:
Expand Down
Loading