How to get real time bid / ask / price from GDAX websocket feed - coinbase-api

The API doc discourages polling on /ticker endpoint, and recommend to use the websocket stream to listen for match message
But the match response only provide a price and a side (sell / buy)
How can I recreate the ticker data (price, ask and bid) from the websocket feed ?
{
“price”: “333.99”,
“size”: “0.193”,
“bid”: “333.98”,
“ask”: “333.99”,
“volume”: “5957.11914015”,
“time”: “2015-11-14T20:46:03.511254Z”
}
The ticker endpoint and the websocket feed both return a 'price', but I guess that it's not the same. Is the price from the ticker endpoint some sort of average over time ?
How can I compute Bid value, Ask value ?

If I use these parameters in the subscribe message:
params = {
"type": "subscribe",
"channels": [{"name": "ticker", "product_ids": ["BTC-EUR"]}]
}
Each time a new trade is executed (and visible on http://www.gdax.com), I get this kind of message from the web socket:
{
u'best_ask': u'3040.01',
u'best_bid': u'3040',
u'last_size': u'0.10000000',
u'price': u'3040.00000000',
u'product_id': u'BTC-EUR',
u'sequence': 2520531767,
u'side': u'sell',
u'time': u'2017-09-16T16:16:30.089000Z',
u'trade_id': 4138962,
u'type': u'ticker'
}
Just after this particular message, I did a get on https://api.gdax.com/products/BTC-EUR/ticker, and I got this:
{
"trade_id": 4138962,
"price": "3040.00000000",
"size": "0.10000000",
"bid": "3040",
"ask": "3040.01",
"volume": "4121.15959844",
"time": "2017-09-16T16:16:30.089000Z"
}
Present data are the same from web socket compared to the get request.
Please find below a complete test script implementing a web socket with this ticker.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Test for websockets."""
from websocket import WebSocketApp
from json import dumps, loads
from pprint import pprint
URL = "wss://ws-feed.gdax.com"
def on_message(_, message):
"""Callback executed when a message comes.
Positional argument:
message -- The message itself (string)
"""
pprint(loads(message))
print
def on_open(socket):
"""Callback executed at socket opening.
Keyword argument:
socket -- The websocket itself
"""
params = {
"type": "subscribe",
"channels": [{"name": "ticker", "product_ids": ["BTC-EUR"]}]
}
socket.send(dumps(params))
def main():
"""Main function."""
ws = WebSocketApp(URL, on_open=on_open, on_message=on_message)
ws.run_forever()
if __name__ == '__main__':
main()

Related

Discord.py bot not running client.loop or tasks in on_ready

I've made a python bot for a small project I am doing. Basically all it does is read a specific set of cells in a google sheet, draw a nice little chart and save it as an image, and then send it to a user in their DMs with their discord id.
However, I've struck a bit of an issue. The bot keeps going offline after 3 minutes or so. No biggie, I'll just change it's status to something funny out of a list every 2 mins, I thought. But, no matter if I do it using a client.loop event or tasks from discord.ext, the loop won't run.
The only times when they do run is when the discord bot throws an error (like losing connection when my wifi decides to have a mind of its own), or when the client.loop event regarding the cells that are being watched is fulfilled.
I have scoured the internet looking for the dumb thing I've done wrong, but can't for the love of me find out why. Can anyone help me? The code is below, with some changes as to not reveal tokens or anything.
import gspread
from oauth2client.service_account import ServiceAccountCredentials
from discord.ext import tasks
import time
import matplotlib.pyplot as plt
import requests
import json
import discord
import random
import asyncio
# Set up the credentials to access the Google Sheets API
scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name(r"json location", scope)
# Authenticate with Google and open the desired sheet
gc = gspread.authorize(credentials)
sheet = gc.open("Spreadsheet name").worksheet('Sheet6')
# Get the initial value of the cell
cell_value = sheet.acell('B2').value
#init some variables
parties = [(20, -16, "royalblue", "BIU"), (-10, -33, "navy", "DXC"), (-10, 9, "gold", "ALP")]
status = [
'you make questionable political choices.',
'you lie on your political compass test.',
'you destroy your political career.',
'you spam Neutral/Unsure.',
'you bank on trickle down economics',
'you expand the government',
'you storm the Capitol',
'you claim you did not have sexual relations with that woman',
'you take secularism as more of a suggestion',
'you change the definition of morally ambiguous'
]
#define some defs
def political_compass(economics, social):
#this thing really is superfluous now but I cba to remove it
return (economics, social)
def plot_result(result, parties=None):
#plot things in a square graph and such
x, y = result
fig, ax = plt.subplots()
ax.scatter(x, y, color="black")
if parties:
for party in parties:
ax.scatter(party[0], party[1], color=party[2], label=party[3])
ax.set_title("TITLE")
ax.set_xlim(-50, 50)
ax.set_ylim(-50, 50)
ax.axhline(y=0, color="black")
ax.axvline(x=0, color="black")
ax.annotate("Economic Left/Right", (35, 0), (0, -5),
textcoords="offset points", ha="center", va="top")
ax.annotate("Progressive/Conservative", (0, 45), (-5, 0),
textcoords="offset points", ha="right", va="center")
if parties:
plt.legend(loc="upper right")
fig.savefig("political_compass.png")
client = discord.Client()
#tasks.loop(minutes=2)
async def status_change():
cur_status = random.choice(status)
print('Current status is: Watching '+cur_status)
await client.change_presence(status=discord.Status.online, activity=discord. Activity(type=discord.ActivityType.watching, name=cur_status))
async def sheet_check():
global cell_value
await client.wait_until_ready()
while not client.is_closed():
time.sleep(1)
current_value = sheet.acell('B2').value
if current_value != cell_value:
print("Cell value has changed from", cell_value, "to", current_value)
cell_value = current_value
# Grabbing info
discord_id = sheet.acell('C2').value
eco_val = float(sheet.acell('D2').value)
soc_val = float(sheet.acell('E2').value)
print(discord_id)
print(eco_val)
print(soc_val)
# make the graph
result = political_compass(eco_val, soc_val)
plot_result(result, parties)
# Upload the image to imgur
client_id = "ID I WON'T SHARE"
image = open("political_compass.png", "rb").read()
headers = {"Authorization": f"Client-ID {client_id}"}
response = requests.post("https://api.imgur.com/3/image", headers=headers, data={"image": image, "type": "file"})
if response.status_code == 200:
response_json = response.json()
print(f"Image uploaded successfully: {response_json['data']['link']}")
else:
print("Failed to upload image")
# Send with discord bot
user = user = await client.fetch_user(discord_id)
await user.send(f"Here is your Political Compass result: {response_json['data']['link']}")
await user.send(f"Additional message")
await user.send(f"other additional message")
t = time.localtime()
current_time = time.strftime("%H:%M:%S", t)
print("Message sent at " + current_time)
await asyncio.sleep(10)
#client.event
async def on_ready():
print(f'Logged in as {client.user}')
await client.change_presence(status=discord.Status.online, activity=discord. Activity(type=discord.ActivityType.watching, name='myself in the mirror.'))
channel = client.get_channel(CHANNEL ID)
await channel.send('Hello, World!')
print('He has awaken')
status_change.start()
client.loop.create_task(sheet_check())
client.run('TOKEN THAT SHOULDNT BE HERE BUT IN A ENV FILE I KNOW')
Thanks in advance for helping.
I tried calling status_change through client.loop and through discord.ext's tasks but neither has resulted in status_change being run on startup and then continuously.

what can i do to send json.dumps as list in django channels?

I want to take the player list from the room and send it to react with the room websocket(django channels). I need a list to handle all users but websocket turns a string like this
this is not array pure string!
[{"model": "api.roomusers", "pk": 396, "fields": {"user_name": "kevin", "score": null}}]
Here is my consumers.py
roomUsers = serializers.serialize(
"json", models.RoomUsers.objects.filter(room=isRoomExist), fields=('user_name', 'score'))
async_to_sync(self.channel_layer.group_send)(
self.room_group_name,
{
'type': 'user_data',
'roomUsers': roomUsers,
}
)
def user_data(self, event):
roomUsers = event["roomUsers"]
async_to_sync(self.send(text_data=json.dumps(
{"type": "user_data", "roomUsers": roomUsers})))
react side
case "user_data":
console.log(data.roomUsers);
console.log(typeof(data.roomUsers));
break;
**react side turn says data.roomUsers is string not an array. **
I am stuck here and I don't have an opinion to solve this situation

How to match input/output with sagemaker batch transform?

I'm using sagemaker batch transform, with json input files. see below for sample input/output files. i have custom inference code below, and i'm using json.dumps to return prediction, but it's not returning json. I tried to use => "DataProcessing": {"JoinSource": "string", }, to match input and output. but i'm getting error that "unable to marshall ..." . I think because , the output_fn is returning array of list or just list and not json , that is why it is unable to match input with output.any suggestions on how should i return the data?
infernce code
def model_fn(model_dir):
...
def input_fn(data, content_type):
...
def predict_fn(data, model):
...
def output_fn(prediction, accept):
if accept == "application/json":
return json.dumps(prediction), mimetype=accept)
raise RuntimeException("{} accept type is not supported by this script.".format(accept))
input file
{"data" : "input line one" }
{"data" : "input line two" }
....
output file
["output line one" ]
["output line two" ]
{
"BatchStrategy": SingleRecord,
"DataProcessing": {
"JoinSource": "string",
},
"MaxConcurrentTransforms": 3,
"MaxPayloadInMB": 6,
"ModelClientConfig": {
"InvocationsMaxRetries": 1,
"InvocationsTimeoutInSeconds": 3600
},
"ModelName": "some-model",
"TransformInput": {
"ContentType": "string",
"DataSource": {
"S3DataSource": {
"S3DataType": "string",
"S3Uri": "s3://bucket-sample"
}
},
"SplitType": "Line"
},
"TransformJobName": "transform-job"
}
json.dumps will not convert your array to a dict structure and serialize it to a JSON String.
What data type is prediction ? Have you tested making sure prediction is a dict?
You can confirm the data type by adding print(type(prediction)) to see the data type in the CloudWatch Logs.
If prediction is a list you can test the following:
def output_fn(prediction, accept):
if accept == "application/json":
my_dict = {'output': prediction}
return json.dumps(my_dict), mimetype=accept)
raise RuntimeException("{} accept type is not supported by this script.".format(accept))
DataProcessing and JoinSource are used to associate the data that is relevant to the prediction results in the output. It is not meant to be used to match the input and output format.

How to extract data from an API and create an array to send to the another API in Jmeter?

Example:
API A:
{
"customer":[
{
"name":"Jane",
"phone":"9999999",
"email":"jane#test.com"
},
{
"name":"John",
"phone":"8888888",
"email":"john#test.com"
},
{
"name":"Joe",
"phone":"7777777",
"email":"Joe#test.com"
}
]
}
Using the JSON extractor, I want to get the names of all the customers
so: Jane, John, Joe
How do I get these values and turn them into an array
[{"name":"Jane", "name":"John", "name":"Joe"}]
And pass it onto the next API?
Note: That it has to be dynamic so API A could show different 2 names or 1 name or more and needs to be adjusted into the array
First of all your [{"name":"Jane", "name":"John", "name":"Joe"}] is not a valid JSON, you can check it yourself:
so I strongly doubt that this is the string you need to generate.
So if you really need to construct this value you can do something like:
Add JSR223 PostProcessor as a child of the request which returns this "customers" data
Put the following code into "Script" area:
def response = new groovy.json.JsonSlurper().parse(prev.getResponseData())
def payload = new StringBuilder()
payload.append('[{')
0.upto(response.customer.size - 1, { index ->
payload.append('"name": "').append(response.customer[index].name).append('"')
if (index != response.customer.size - 1) {
payload.append(',')
}
})
payload.append('}]')
vars.put('payload', payload as String)
Refer the generated value as ${payload} where required
Demo:
More information:
JsonSlurper
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It

What is the meaning of []json.Rawmessage

What is the meaning of []json.Rawmessage. Its within this structure here:
type Request struct {
Jsonrpc string `json:"jsonrpc"`
Method string `json:"method"`
Params []json.RawMessage `json:"params"`
ID interface{} `json:"id"`
}
I know its a slice of type json. I do not understand what the .RawMessage is referring to. I tried looking it up in both golang tour and my golang book. Also ultimately I know Params is type []json.Rawmessage being bundled into another type called Request
Futhermore:
What is going on with these segments json:"jsonrpc". Is a string literal somehow being attached to var? Again this is not in golang tour or my golang book. Thanks for your time.
[] is defining a slice
json is the package import name (from your import statement above)
RawMessage is the type within the package. In this case for a []byte type.
json:"params" is a field tag. The json package reads it with reflection and determines what name to use for json.
Most of the time time you need to look in to the package doc rather than some book and the online tour.
The json:"jsonrpc" is a struct tag.
For this specific case the json.Marshal function will read that struct tag and return the JSON field name with the given value. For more about "encoding/json" struct tags usage: https://golang.org/pkg/encoding/json/#Marshal
For RawMessage, you can read more about it here https://golang.org/pkg/encoding/json/#RawMessage
type RawMessage []byte
Normally I use it when having a "generic" kind of JSON object that don't need to be process right the way (maybe I just send them to another service, or I will unmarshall it later depending on conditions). For example (I will use your Request struct):
jsonString := `[
{
"id": 123,
"method": "getSomething",
"params": [{"A": 1, "B": 2}]
}
{
"id": 123,
"method": "getSomethingElse",
"params": [{"C": 1, "D": 2}]
}
]`
With this processing code:
var requests []Request
json.Unmarshal([]byte(jsonString), &requests)
// if no error, you will have 2 items in requests
// requests[0].Params[0] is equal to []byte(`{"A": 1, "B": 2}`)
// requests[1].Params[0] is equal to []byte(`{"C": 1, "D": 2}`)
for _, req := range requests {
if req.Method == "getSomething" {
justProxyThisRequestToAnotherService(req)
} else if req.Method == "getSomethingElse" {
var params []map[string]int
json.Unmarshal(req.Params, &params)
// then do something with params
}
}

Resources