I was Trying to to Calculate initial embedding of all data frame which is first step to implement my GNN which is Heterogeneous in nature.I have used twitter msg data set in the task I loaded it in following way
load_path = '/Users/hemangjiwnani/Desktop/Projects/Paper1/KPGNN/datasets/Twitter/'
save_path = '/Users/hemangjiwnani/Desktop/Projects/Paper1/KPGNN/datasets/Twitter/'
# load dataset`enter code here`
p_part1 = load_path + '68841_tweets_multiclasses_filtered_0722_part1.npy'
p_part2 = load_path + '68841_tweets_multiclasses_filtered_0722_part2.npy'
#"./datasets/Twitter/68841_tweets_multiclasses_filtered_0722_part1.npy"
df_np_part1 = np.load(p_part1, allow_pickle=True)
df_np_part2 = np.load(p_part2, allow_pickle=True)
Then I have created a data frame of the same with the help of following code
df_np = np.concatenate((df_np_part1, df_np_part2), axis = 0) #Axis = 0 means horizontal
print("Loaded data.")
df = pd.DataFrame(data=df_np, columns=["event_id", "tweet_id", "text", "user_id", "created_at", "user_loc",\
"place_type", "place_full_name", "place_country_code", "hashtags", "user_mentions", "image_urls", "entities",
"words", "filtered_words", "sampled_words"])
print("Data converted to dataframe.")
print(df.shape)
print(df.head(5))
Which was having following output
Loaded data.
Data converted to dataframe.
(68841, 16)
event_id ... sampled_words
0 0 ... []
1 0 ... []
2 0 ... []
3 0 ... []
4 0 ... []
[5 rows x 16 columns]
This Function Below is raising a error while returning
def documents_to_features(df):
nlp = spacy.load("en_core_web_sm"
#nlp = en_core_web_lg.load()
features = df.filtered_words.apply(lambda x: nlp(' '.join(x)).vector).values
-->return np.stack(features, axis=0)
ERROR
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-12-2e8bc5e83009> in <module>()
----> 1 d_features = documents_to_features(df)
2 print("Document features generated.")
3 t_features = df_to_t_features(df)
4 print("Time features generated.")
5 combined_features = np.concatenate((d_features, t_features), axis=1)
1 frames
<ipython-input-9-b772a7744232> in documents_to_features(df)
3 #nlp = en_core_web_lg.load()
4 features = df.filtered_words.apply(lambda x: nlp(' '.join(x)).vector).values
----> 5 return np.stack(features, axis=0)
<__array_function__ internals> in stack(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/numpy/core/shape_base.py in stack(arrays, axis, out)
425 shapes = {arr.shape for arr in arrays}
426 if len(shapes) != 1:
--> 427 raise ValueError('all input arrays must have the same shape')
428
429 result_ndim = arrays[0].ndim + 1
ValueError: all input arrays must have the same shape
I tried to use the dataset API loaded with my own data to train the hunggingface model.
This is my code:
train_data= datasets.load_dataset('csv', data_files="/gdrive/MyDrive/project/train.csv")
test_data= datasets.load_dataset('csv', data_files="/gdrive/MyDrive/project/test.csv")
train_data
DatasetDict({
train: Dataset({
features: ['Post', 'Label'],
num_rows: 174
})
})
But in the trainer, I get this error message. What happens to my dataset. I do not find any error. Could you help me? Thank you!
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_data,
eval_dataset=test_data
)
trainer.train()
KeyError Traceback (most recent call last)
<ipython-input-63-3435b262f1ae> in <module>()
----> 1 trainer.train()
5 frames
/usr/local/lib/python3.7/dist-packages/datasets/dataset_dict.py in __getitem__(self, k)
44 suggested_split = available_suggested_splits[0] if available_suggested_splits else list(self)[0]
45 raise KeyError(
---> 46 f"Invalid key: {k}. Please first select a split. For example: "
47 f"`my_dataset_dictionary['{suggested_split}'][{k}]`. "
48 f"Available splits: {sorted(self)}"
KeyError: "Invalid key: 0. Please first select a split. For example: `my_dataset_dictionary['train'][0]`. Available splits: ['train']"
I am trying to run a fairly simple MCMC sample for some time series data. I believe I am including all the required args, but I'm still getting an error.
The library versions:
tensorflow==1.14.0
tensorflow-proability==0.7.0
I tried rolling back to tfp 0.6.0 and got a matmul error. I tried pushing forward to tf nightly and got the same error as below.
The code
import tensorflow as tf
import tensorflow_probability as tfp
tfd = tfp.distributions
tf.enable_eager_execution()
rate_prior = tfd.Exponential(1./mean_users)
users_before = [...] # bogus positive ints
users_after = [...] # bogus positive ints
def unnormalized_log_prob(rate_before, rate_after):
users_before_prior = tfd.Poisson(rate_before)
users_after_prior = tfd.Poisson(rate_after)
return (rate_prior.log_prob(rate_before)
+ rate_prior.log_prob(rate_after)
+ tf.reduce_sum(users_before_prior.log_prob(users_before))
+ tf.reduce_sum(users_after_prior.log_prob(users_after))
)
bijectors = [tfp.bijectors.Exp, tfp.bijectors.Exp]
hmc = tfp.mcmc.TransformedTransitionKernel(
tfp.mcmc.HamiltonianMonteCarlo(
target_log_prob_fn=unnormalized_log_prob,
step_size=10.,
num_leapfrog_steps=3,
),
bijectors
)
states = tfp.mcmc.sample_chain(
num_results=10000,
current_state=[tf.ones(2) * mean_users],
kernel=hmc,
trace_fn=None
)
This returns an error.
.../tensorflow_probability/python/mcmc/transformed_kernel.py in <listcomp>(.0)
71 def fn(state_parts):
72 return [b.inverse(sp)
---> 73 for b, sp in zip(bijector, state_parts)]
74 return fn
75
TypeError: inverse() missing 1 required positional argument: 'y'`
Try
bijectors = [tfp.bijectors.Exp(), tfp.bijectors.Exp()]]
I use Anaconda and gdsCAD and get an error when all packages are installed correctly.
Like explained here: http://pythonhosted.org/gdsCAD/
TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('S32') dtype('S32') dtype('S32')
My imports look like this (In the end I imported everything):
import numpy as np
from gdsCAD import *
import matplotlib.pyplot as plt
My example code looks like this:
something = core.Elements()
box=shapes.Box( (5,5),(1,5),0.5)
core.default_layer = 1
core.default_colors = 2
something.add(box)
something.show()
My error message looks like this:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-2f90b960c1c1> in <module>()
31 puffer_wafer = shapes.Circle((0.,0.), puffer_wafer_radius, puffer_line_thickness)
32 bp.add(puffer_wafer)
---> 33 bp.show()
34 wafer = shapes.Circle((0.,0.), wafer_radius, wafer_line_thickness)
35 bp.add(wafer)
C:\Users\rpilz\AppData\Local\Continuum\Anaconda2\lib\site-packages\gdscad-0.4.5-py2.7.egg\gdsCAD\core.pyc in _show(self)
80 ax.margins(0.1)
81
---> 82 artists=self.artist()
83 for a in artists:
84 a.set_transform(a.get_transform() + ax.transData)
C:\Users\rpilz\AppData\Local\Continuum\Anaconda2\lib\site-packages\gdscad-0.4.5-py2.7.egg\gdsCAD\core.pyc in artist(self, color)
952 art=[]
953 for p in self:
--> 954 art+=p.artist()
955 return art
956
C:\Users\rpilz\AppData\Local\Continuum\Anaconda2\lib\site-packages\gdscad-0.4.5-py2.7.egg\gdsCAD\core.pyc in artist(self, color)
475 poly = lines.buffer(self.width/2.)
476
--> 477 return [descartes.PolygonPatch(poly, lw=0, **self._layer_properties(self.layer))]
478
479
C:\Users\rpilz\AppData\Local\Continuum\Anaconda2\lib\site-packages\gdscad-0.4.5-py2.7.egg\gdsCAD\core.pyc in _layer_properties(layer)
103 # Default colors from previous versions
104 colors = ['k', 'r', 'g', 'b', 'c', 'm', 'y']
--> 105 colors += matplotlib.cm.gist_ncar(np.linspace(0.98, 0, 15))
106 color = colors[layer % len(colors)]
107 return {'color': color}
TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('S32') dtype('S32') dtype('S32')
The gdsCAD has been a pain from shapely install to this plotting issue.
This issue is because of wrong datatype being passed to colors function. It can be solved by editing the following line in core.py
colors += matplotlib.cm.gist_ncar(np.linspace(0.98, 0, 15))
to
colors += list(matplotlib.cm.gist_ncar(np.linspace(0.98, 0, 15)))
If you dont know where the core.py is located. Just type in:
from gdsCAD import *
core
This will give you the path of core.py file. Good luck !
Well first, I'd ask that you please include actual code, as the 'example code' in the file is obviously different based on the traceback. When debugging, the details matter, and I need to be able to actually run the code.
You obviously have a data type problem. Chances are pretty good it's in the variables here:
puffer_wafer = shapes.Circle((0.,0.), puffer_wafer_radius, puffer_line_thickness)
I had the same error thrown when I was running a call to Pandas. I changed the data to str(data) and the code worked.
I don't know if this helps I am fairly new to this myself, but I had a similar error and found that it is due to a type casting issue as suggested by previous answer. I can't see from the example in the question exactly what you are trying to do. Below is a small example of my issue and solution. My code is making a simple Random Forest model using scikit learn.
Here is an example that will give the error and it is caused by the third to last line, concatenating the results to write to file.
import scipy
import math
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn import preprocessing, metrics, cross_validation
Data = pd.read_csv("Free_Energy_exp.csv", sep=",")
Data = Data.fillna(Data.mean()) # replace the NA values with the mean of the descriptor
header = Data.columns.values # Ues the column headers as the descriptor labels
Data.head()
test_name = "Test.csv"
npArray = np.array(Data)
print header.shape
npheader = np.array(header[1:-1])
print("Array shape X = %d, Y = %d " % (npArray.shape))
datax, datay = npArray.shape
names = npArray[:,0]
X = npArray[:,1:-1].astype(float)
y = npArray[:,-1] .astype(float)
X = preprocessing.scale(X)
XTrain, XTest, yTrain, yTest = cross_validation.train_test_split(X,y, random_state=0)
# Predictions results initialised
RFpredictions = []
RF = RandomForestRegressor(n_estimators = 10, max_features = 5, max_depth = 5, random_state=0)
RF.fit(XTrain, yTrain) # Train the model
print("Training R2 = %5.2f" % RF.score(XTrain,yTrain))
RFpreds = RF.predict(XTest)
with open(test_name,'a') as fpred :
lenpredictions = len(RFpreds)
lentrue = yTest.shape[0]
if lenpredictions == lentrue :
fpred.write("Names/Label,, Prediction Random Forest,, True Value,\n")
for i in range(0,lenpredictions) :
fpred.write(RFpreds[i]+",,"+yTest[i]+",\n")
else :
print "ERROR - names, prediction and true value array size mismatch."
This leads to an error of;
Traceback (most recent call last):
File "min_example.py", line 40, in <module>
fpred.write(RFpreds[i]+",,"+yTest[i]+",\n")
TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('S32') dtype('S32') dtype('S32')
The solution is to make each variable a str() type on the third to last line then write to file. No other changes to then code have been made from the above.
import scipy
import math
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn import preprocessing, metrics, cross_validation
Data = pd.read_csv("Free_Energy_exp.csv", sep=",")
Data = Data.fillna(Data.mean()) # replace the NA values with the mean of the descriptor
header = Data.columns.values # Ues the column headers as the descriptor labels
Data.head()
test_name = "Test.csv"
npArray = np.array(Data)
print header.shape
npheader = np.array(header[1:-1])
print("Array shape X = %d, Y = %d " % (npArray.shape))
datax, datay = npArray.shape
names = npArray[:,0]
X = npArray[:,1:-1].astype(float)
y = npArray[:,-1] .astype(float)
X = preprocessing.scale(X)
XTrain, XTest, yTrain, yTest = cross_validation.train_test_split(X,y, random_state=0)
# Predictions results initialised
RFpredictions = []
RF = RandomForestRegressor(n_estimators = 10, max_features = 5, max_depth = 5, random_state=0)
RF.fit(XTrain, yTrain) # Train the model
print("Training R2 = %5.2f" % RF.score(XTrain,yTrain))
RFpreds = RF.predict(XTest)
with open(test_name,'a') as fpred :
lenpredictions = len(RFpreds)
lentrue = yTest.shape[0]
if lenpredictions == lentrue :
fpred.write("Names/Label,, Prediction Random Forest,, True Value,\n")
for i in range(0,lenpredictions) :
fpred.write(str(RFpreds[i])+",,"+str(yTest[i])+",\n")
else :
print "ERROR - names, prediction and true value array size mismatch."
These examples are from a larger code so I hope the examples are clear enough.
I have a Dictionary model defined in Django (1.6.5). One method (called get_topentities) returns the top names in my dictionary (entity names are defined by Entity model):
def get_topentities(self,n):
entities = self.entity_set.select_related().filter(in_dico=True,table_type=0).order_by("rank")[0:n]
return entities
When I call the function (say with n=2), it returns the top 2 elements but I cannot access the second one because of this "not enough arguments to format string" error:
In [5]: d = Dictionary.objects.get(code='USA')
In [6]: top2 = d.get_topentities(2)
In [7]: top2
Out[7]: [<Entity: BARACK OBAMA>, <Entity: GOVERNMENT>]
In [8]: top2[0]
Out[8]: <Entity: BARACK OBAMA>
In [9]: top2[1]
.
.
/usr/local/lib/python2.7/dist-packages/django_pyodbc/compiler.pyc in as_sql(self, with_limits, with_col_aliases)
172 # Lop off ORDER... and the initial "SELECT"
173 inner_select = _remove_order_limit_offset(raw_sql)
--> 174 outer_fields, inner_select = self._alias_columns(inner_select)
175
176 order = _get_order_limit_offset(raw_sql)[0]
/usr/local/lib/python2.7/dist-packages/django_pyodbc/compiler.pyc in _alias_columns(self, sql)
339
340 # store the expanded paren string
--> 341 parens[key] = buf% parens
342 #cannot use {} because IBM's DB2 uses {} as quotes
343 paren_buf[paren_depth] += '(%(' + key + ')s)'
TypeError: not enough arguments for format string
In [10]:
My server backend is MSSQL and I'm using pyodbc as the database driver. If I try the same on a PC with engine sqlserver_ado, it works. Can someone help?
Regards,
Patrick