sorry for my annoying questions again.
Having a bit of trouble with some code. The task is to write a function that takes a nested list of currency conversions and turn it into a table (I've attached some pictures for clarification)
(could only attach one image, this is the nested list it converts)
[[10, 9.6, 7.5, 6.7, 4.96], [20, 19.2, 15.0, 13.4, 9.92], [30, 28.799999999999997, 22.5, 20.1, 14.879999999999999], [40, 38.4, 30.0, 26.8, 19.84], [50, 48.0, 37.5, 33.5, 24.8], [60, 57.599999999999994, 45.0, 40.2, 29.759999999999998], [70, 67.2, 52.5, 46.900000000000006, 34.72], [80, 76.8, 60.0, 53.6, 39.68], [90, 86.39999999999999, 67.5, 60.300000000000004, 44.64], [100, 96.0, 75.0, 67.0, 49.6]]
I've got the Header column for the table to work fine.
I'm having issues when I'm trying to iterate over each sublist in the nested list, convert it to a string (and two decimal places) and with a tab between each entry.
the code I've got so far is:
def printTable(cur):
list2 = makeTable(cur)
lst1 = Extract(cur)
lst1.insert(0, "$NZD")
lne1 = "\t".join(lst1)
print(lne1)
list=map(str,list2)
print(list2)
for list in list2:
for elem in list:
linelem = "\t".join(elem)
print(linelem)
printTable(cur)
(Note: The first function I call and assign to list2 is what generates the data/nested list)
I've tried playing around a bit but I keep coming up with different error messages trying to convert each sublist to a string.
Thank you all for your help!![enter image description here][1]
Try using this method;
import pandas as pd
import csv
with open("out.csv", "w", newline="") as f:
writer = csv.writer(f)
writer.writerows(a)
pd.read_csv("out.csv", header = [col1, col2, col3, col4, col5])
I am trying to use zeppelin to plot a realtime graph. I have doing a sentiment analysis on tweets on per minute . I am able to query statically and plot a graph. But i would like this to be done dynamically. I am new to zeppelin and do not have much knowledge about angularJS. What should be the correct approach to this problem?
val final_score=uni_join.map{case((year,month,day,hour,minutes),(tweet_count,sentiment))=>(year, month, day, hour, minutes(sentiment/tweet_count).ceil)}
final_score.saveToCassandra("twitter", "score",writeConf = WriteConf(ttl = TTLOption.constant(1000)))
final_score.foreachRDD(score => {
val rowRDD =score.map{case(year,month,day,hour,minutes,sentiment) =>(year,month,day,hour,minutes,sentiment) }
val tempDF = sqlContext.createDataFrame(rowRDD)
z.angularBindGlobal("stream", parsed) //to bind parsed to stream.
tempDF.registerTempTable("realTimeTable")
})
Doing a query on the above table , i am able to get the graph. But i would like to dynamically update the graph every minute in order to keep in sync with the sentiment score .
Thanks prior.
[update] the angular part for the zeppelin notebook are as follows:
%angular
<div id="graph" style="height: 100%; width: 100%">
<canvas id="myChart" width="400" height="400"></canvas>
<div id="legendDiv"></div>
</div>
<script>
function initMap() {
var colorList = ["#fde577", "#ff6c40", "#c72a40", "#520833", "#a88399"]
var el = angular.element($('#stream'));
console.log("El is "+el) //returns el as object
angular.element(el).ready(function() {
console.log('Hello')
window.locationWatcher =el.$scope.$watch('stream', function(new, old){
console.log('changed');}, true)})
</script>
But running this code keeps returning the following error.
vendor.js:29 jQuery.Deferred exception: Cannot read property '$watch' of undefined TypeError: Cannot read property '$watch' of undefined
The spark version that i am using is 1.6
And Zeppelin is 0.6
spark-highcharts since version 0.6.3 support Spark Structured Streaming.
For a structuredDataFrame after aggregation, with the following code in one Zeppelin paragraph. The OutputMode can be either append or complete depends how the structureDataFrame is aggregated.
import com.knockdata.spark.highcharts._
import com.knockdata.spark.highcharts.model._
val query = highcharts(
structuredDataFrame.seriesCol("country")
.series("x" -> "year", "y" -> "stockpile")
.orderBy(col("year")), z, "append")
And the following code in the next paragraph. The chart in this paragraph will be updated when there are new data coming to the structureDataFrame.
StreamingChart(z)
Run following code to stop update the chart.
query.stop()
Here is the example generate structureDataFrame.
spark.conf.set("spark.sql.streaming.checkpointLocation","/usr/zeppelin/checkpoint")
case class NuclearStockpile(country: String, stockpile: Int, year: Int)
val USA = Seq(0, 0, 0, 0, 0, 6, 11, 32, 110, 235, 369, 640,
1005, 1436, 2063, 3057, 4618, 6444, 9822, 15468, 20434, 24126,
27387, 29459, 31056, 31982, 32040, 31233, 29224, 27342, 26662,
26956, 27912, 28999, 28965, 27826, 25579, 25722, 24826, 24605,
24304, 23464, 23708, 24099, 24357, 24237, 24401, 24344, 23586,
22380, 21004, 17287, 14747, 13076, 12555, 12144, 11009, 10950,
10871, 10824, 10577, 10527, 10475, 10421, 10358, 10295, 10104).
zip(1940 to 2006).map(p => NuclearStockpile("USA", p._1, p._2))
val USSR = Seq(0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
5, 25, 50, 120, 150, 200, 426, 660, 869, 1060, 1605, 2471, 3322,
4238, 5221, 6129, 7089, 8339, 9399, 10538, 11643, 13092, 14478,
15915, 17385, 19055, 21205, 23044, 25393, 27935, 30062, 32049,
33952, 35804, 37431, 39197, 45000, 43000, 41000, 39000, 37000,
35000, 33000, 31000, 29000, 27000, 25000, 24000, 23000, 22000,
21000, 20000, 19000, 18000, 18000, 17000, 16000).
zip(1940 to 2006).map(p => NuclearStockpile("USSR/Russia", p._1, p._2))
input.addData(USA.take(30) ++ USSR.take(30))
val structureDataFrame = input.toDF
And the following code can be simulate to update the chart. The chart will be updated when the following code run.
input.addData(USA.drop(30) ++ USSR.drop(30))
NOTE: The example using Zeppelin 0.6.2 and Spark 2.0
NOTE: Please check the highcharts license for commercial usage
Good day,
I'm getting the following error: is not JSON serializable
but I'm not sure why am I getting it. Everything was working fine until I decided to start making use of sessions to fire up my user cart adding and removal of items
This is my view:
def add_or_update_cart(request, slug):
request.session.set_expiry(180)
new_total = 0.00
try:
# check that session exists
the_cart_id = request.session['cart_id']
except:
new_cart_id = Cart()
new_cart_id.save()
request.session['cart_id'] = new_cart_id
the_cart_id = new_cart_id.id
cart = Cart.objects.get(id=the_cart_id)
try:
product = Product.objects.get(slug=slug)
except Product.DoesNotExist:
pass
except:
pass
if not product in cart.products.all():
cart.products.add(product)
else:
cart.products.remove(product)
for item in cart.products.all():
new_total += float(item.price)
request.session['items_total'] = cart.products.count()
cart.total = new_total
cart.save()
print(cart.products.count())
return HttpResponseRedirect(reverse('cart:cart'))
Models:
class Cart(models.Model):
products = models.ManyToManyField(Product, null=True, blank=True)
total = models.DecimalField(max_digits=100, decimal_places=2, default=0.00)
timestamp = models.DateTimeField(auto_now_add=True, auto_now=False)
updated = models.DateTimeField(auto_now_add=False, auto_now=True)
cart_status = models.BooleanField(default=False)
def __str__(self):
return '%s' % self.id
def item_name(self):
return " ".join([str(p) for p in self.product.all()])
and in my template:
<li role="presentation">Cart <span class="badge">{{ request.session.items_total }}</span></li>
The traceback:
Internal Server Error: /my-cart/puma/
Traceback (most recent call last):
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/core/handlers/base.py", line 235, in get_response
response = middleware_method(request, response)
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/contrib/sessions/middleware.py", line 50, in process_response
request.session.save()
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/contrib/sessions/backends/db.py", line 82, in save
obj = self.create_model_instance(data)
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/contrib/sessions/backends/db.py", line 68, in create_model_instance
session_data=self.encode(data),
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/contrib/sessions/backends/base.py", line 88, in encode
serialized = self.serializer().dumps(session_dict)
File "/home/drcongo/.virtualenvs/eCommerce/lib/python3.4/site-packages/django/core/signing.py", line 95, in dumps
return json.dumps(obj, separators=(',', ':')).encode('latin-1')
File "/usr/lib/python3.4/json/__init__.py", line 237, in dumps
**kw).encode(obj)
File "/usr/lib/python3.4/json/encoder.py", line 192, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.4/json/encoder.py", line 250, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python3.4/json/encoder.py", line 173, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <Cart: 31> is not JSON serializable
I will appreciate any help on this.
The error arises when Django tries to serialize a model instance here
request.session['cart_id'] = new_cart_id
new_cart_id is a model instance and cannot be serialized.
It seems you wanted to assign the primary key of the instance to the key cart_id.
request.session['cart_id'] = new_cart_id.id
After searching on the internet, I know how to create a 1*j struct array. For example,
>> patient(1).name = 'John Doe';
patient(1).billing = 127.00;
patient(1).test = [79, 75, 73; 180, 178, 177.5; 220, 210, 205];
>>
>> patient(2).name = 'Ann Lane';
patient(2).billing = 28.50;
patient(2).test = [68, 70, 68; 118, 118, 119; 172, 170, 169];
My question is: how to create a j*1 struct array? Thanks so much for your time and attention.
There are two solutions:
You transpose your structure array afterwards:
patient = patient';
You index your structure array with two integers (first=row, second=column):
patient(j,1).name = 'John Doe';
Best,