I have created a Volttron agent and trying to use pandas library.
But I am getting error of pandas not being installed, while its there.
ERROR:volttron.platform.packaging:b'Traceback (most recent call last):\n File "setup.py", line 11, in <module>
_temp = __import__(agent_module, globals(), locals(), [\'__version__\'], 0)
File "/tmp/tmpd_5srlls/pkg/weather/agent.py", line 12, in <module>
import pandas as pd
File "/home/pi/volttron/env/lib/python3.7/site-packages/pandas/__init__.py", line 17, in <module>
"Unable to import required dependencies:\\n" + "\\n".join(missing_dependencies)
ImportError: Unable to import required dependencies:\nnumpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was\ninstalled.\n\nWe have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html\n\nPlease note and check the following:\n\n * The Python version is: Python3.7 from "/home/pi/volttron/env/bin/python"
* The NumPy version is: "1.19.5"\n\nand make sure that they are the versions you expect.\nPlease carefully study the documentation linked above for further help.Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory\n\n'
Traceback (most recent call last):
File "scripts/install-agent.py", line 340, in <module>
if not os.path.isfile(opts.package):
File "/usr/lib/python3.7/genericpath.py", line 30, in isfile
st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
The shared library cannot be located. Please try:
sudo apt-get install libatlas-base-dev
Related
I will try to be as short as possible.
I ran a Neural prophet forecasting job on multiple products
Task 'model_selection': Exception encountered during task execution!
Traceback (most recent call last):
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/builds/-/--prefect-workflows/workflows/worker_flow.py", line 108, in model_selection
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/python_translation/model_selection_master.py", line 483, in run_model_selection
) = cross_validate_neuralprophet(
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/python_translation/models/NeuralProphet.py", line 169, in cross_validate_neuralprophet
train = NeuralProphet_model.fit(df=df_train, freq="W-MON")
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/forecaster.py", line 592, in fit
metrics_df = self._train(df_dict, progress=progress)
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/forecaster.py", line 1806, in _train
loader = self._init_train_loader(df_dict)
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/forecaster.py", line 1572, in _init_train_loader
self.config_normalization.init_data_params(
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/configure.py", line 41, in init_data_params
self.local_data_params, self.global_data_params = df_utils.init_data_params(
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/df_utils.py", line 260, in init_data_params
global_data_params = data_params_definition(
File "/root/.cache/pypoetry/virtualenvs/--py3.8/lib/python3.8/site-packages/neuralprophet/df_utils.py", line 176, in data_params_definition
data_params[covar] = get_normalization_params(
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/df_utils.py", line 300, in get_normalization_params
norm_type = auto_normalization_setting(array)
File "/root/.cache/pypoetry/virtualenvs/--prefect-workflows-9TtSrW0h-py3.8/lib/python3.8/site-packages/neuralprophet/df_utils.py", line 290, in auto_normalization_setting
raise ValueError
ValueError
Describe the bug
Ran a forecasting job ... and it raised a ValueError without any additional mentions.
To Reproduce
I really do not know. It was a Prefect job that I ran over 200 products. And I have no idea why it failed.
Expected behavior
I expected it to forecast without returning an error.
What actually happens
It crashes with a ValueError
Screenshots
Printouts are above.
Environement (please complete the following information):
Python environment: 3.8.10
NeuralProphet version: neuralprophet 0.3.2, installed from PYPI with pip install neuralprophet
Additional context
These are scheduled as a Prefect workflow. Hence I do not run things manually. Around 150 products ran without any issues. And this returned a ValueError.
I am trying to import some files from my Google Cloud Storage bucket to a Google Colab Notebook. However, I face the following error using the snippet suggested by google.
Code:
!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
# Authenticate and create the PyDrive client.
# This only needs to be done once per notebook.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
# Download a file based on its file ID.
#
# A file ID looks like: laggVyWshwcyP6kEI-y_W3P8D26sz
file_id = '1NKNtpuP95NKNMTVao6qt8G-OJs_xmbnC'
downloaded = drive.CreateFile({'id': file_id})
print('Downloaded content "{}"'.format(downloaded.GetContentString()))
Error:
W0331 16:31:52.011976 139994604533632 __init__.py:44] file_cache is unavailable when using oauth2client >= 4.0.0
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/googleapiclient/discovery_cache/__init__.py", line 36, in autodetect
from google.appengine.api import memcache
ModuleNotFoundError: No module named 'google.appengine'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in <module>
from oauth2client.contrib.locked_file import LockedFile
ModuleNotFoundError: No module named 'oauth2client.contrib.locked_file'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in <module>
from oauth2client.locked_file import LockedFile
ModuleNotFoundError: No module named 'oauth2client.locked_file'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/googleapiclient/discovery_cache/__init__.py", line 41, in autodetect
from . import file_cache
File "/usr/local/lib/python3.6/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in <module>
'file_cache is unavailable when using oauth2client >= 4.0.0')
ImportError: file_cache is unavailable when using oauth2client >= 4.0.0
Downloaded content "{
"attention_probs_dropout_prob": 0.1,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"initializer_range": 0.02,
"intermediate_size": 3072,
"max_position_embeddings": 512,
"num_attention_heads": 12,
"num_hidden_layers": 12,
"type_vocab_size": 2,
"vocab_size": 28996
}
"
I understand that I have some modules missing and I need to install them. However, I tried
!export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"
!echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
!apt update && apt install google-cloud-sdk
And faced the following error:
deb http://packages.cloud.google.com/apt main
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1326 100 1326 0 0 53040 0 --:--:-- --:--:-- --:--:-- 53040
OK
E: Malformed entry 1 in list file /etc/apt/sources.list.d/google-cloud-sdk.list (Component)
E: The list of sources could not be read.
I am new to Google Colab Notebooks and while I understand that it's a simple interactive notebook I am not able to get the hang of this.
Thanks,
It is a bug in googleapiclient, the code bellow will sove it.
import logging
logging.getLogger('googleapiclient.discovery_cache').setLevel(logging.ERROR)
Ref : Github Reference Error
I am getting the following error when trying to run the nosetest from my GAE project:
nosetests --nologcapture --with-gae --without-sandbox --gae-lib-root=/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine
but I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.3.4', 'console_scripts', 'nosetests')()
File "/Library/Python/2.7/site-packages/nose/core.py", line 121, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Library/Python/2.7/site-packages/nose/core.py", line 145, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.7/site-packages/nose/config.py", line 346, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 284, in configure
cfg(options, config)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 99, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 167, in simple
result = meth(*arg, **kw)
File "/Library/Python/2.7/site-packages/nosegae.py", line 87, in configure
from google.appengine.tools import old_dev_appserver as dev_appserver
ImportError: cannot import name old_dev_appserver
The sys.path reads:
'/Users/dsinha/Downloads/eclipse/plugins/org.python.pydev_3.9.0.201411111611/pysrc', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/antlr3', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/fancy_urllib', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/ipaddr', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/rsa', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1_modules', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/simplejson', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/django-1.4', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/goo...
so the app engine libraries should be getting pulled in. Actually its first failing to pull in dev_appserver, and then trying and failing to pull old_dev_appserver
The dir inside the appengine/lib file is:
bash-3.2$ ls
__init__.py appcfg.py backends_xml_parser.py dispatch_xml_parser.py handler_generator.py php_cli.py value_mixin.pyc
__init__.pyc appcfg_java.py boolean_action.py docker handler_generator.pyc queue_xml_parser.py web_xml_parser.py
adaptive_thread_pool.py appengine_rpc.py boolean_action.pyc dos_xml_parser.py jarfile.py queue_xml_parser.pyc web_xml_parser.pyc
api_server.py appengine_rpc.pyc bulkload_client.py download_appstats.py java_quickstart.py remote_api_shell.py xml_parser_utils.py
app_engine_config_exception.py appengine_rpc_httplib2.py bulkloader.py endpointscfg.py java_quickstart.pyc requeue.py xml_parser_utils.pyc
app_engine_config_exception.pyc augment_mimetypes.py cron_xml_parser.py gen_protorpc.py java_utils.py sdk_update_checker.py yaml_translator.py
app_engine_web_xml_parser.py augment_mimetypes.pyc dev-channel-js.js handler.py java_utils.pyc sdk_update_checker.pyc yaml_translator.pyc
app_engine_web_xml_parser.pyc backends_conversion.py devappserver2 handler.pyc os_compat.py value_mixin.py
bash-3.2$ pwd
/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/google/appengine/tools
I also tried to find the modules available inside the google.appengine.tools package:
>>> import pkgutil
>>> [name for _, name, _ in pkgutil.iter_modules(['testpkg'])]
[]
This problem started occuring after I upgraded to App Engine 1.9.10 (to use the async search features). In a problem that I think is related, when I try to run the debug server from PyDev, it just silently terminated on any page request (localhost:8080).
Running dev_appserver . from the command line works fine though.
Nose-GAE broke with App Engine 1.9.17: https://github.com/Trii/NoseGAE/issues/6
Downgrading to 1.9.15 made the problem go away temporarily while waiting for the issue to be resolved by nose-gae
Trying to import testbed from GAE SDK 1.95 with Python2.7.8 on osX Maverics 10.9.5
and I'm getting a InvalidCertificateException error.
from google.appengine.ext import testbed
File "/usr/local/google_appengine/google/appengine/ext/testbed/__init__.py", line 120, in <module>
from google.appengine.api import urlfetch_stub
File "/usr/local/google_appengine/google/appengine/api/urlfetch_stub.py", line 34, in <module>
_fancy_urllib_InvalidCertException = fancy_urllib.InvalidCertificateException
AttributeError: 'module' object has no attribute 'InvalidCertificateException'
I looked at the fancy_url module and the InvalidCertificateException class is there, so I don't understand why it's not importing.
Apparently others have had the same error, so I attempt to correct it by deleting: urlfetch_cacerts.txt AND cacerts.txt from:
GoogleAppEngineLauncher/Contents/Resources/GoogleAppEngineDefault.bundle/Contents/Resources/google_appengine/lib/cacerts/
Apparently the GAE installer creates a nested directory, this was fixed by copying the contents in:
cd /usr/local/google_appengine/lib
cp fancy_urllib/fancy_urllib/__init__.py fancy_urllib/__init__.py
This is how the module is incorrectly structured, it looks like these 2 init.py files are duplicate:
/usr/local/google_appengine/lib/fancy_urllib/__init__.py # this file is empty
/usr/local/google_appengine/lib/fancy_urllib/fancy_urllib/__init__.py # this file contains the functions.
FIXED THE ERROR
I'm playing around with django-nonrel and google app engine when I get the following error trying to create a new app.
Traceback (most recent call last):
File "./manage.py", line 2, in <module>
from django.core.management import execute_manager
File "/home/levi/Projects/DoneBox/django/__init__.py", line 14
if svn_rev != u'SVN-unknown':
^
SyntaxError: invalid syntax
The file that caused the exception is included below.
VERSION = (1, 3, 0, 'final', 0)
def get_version():
version = '%s.%s' % (VERSION[0], VERSION[1])
if VERSION[2]:
version = '%s.%s' % (version, VERSION[2])
if VERSION[3:] == ('alpha', 0):
version = '%s pre-alpha' % version
else:
if VERSION[3] != 'final':
version = '%s %s %s' % (version, VERSION[3], VERSION[4])
from django.utils.version import get_svn_revision
svn_rev = get_svn_revision()
if svn_rev != u'SVN-unknown':
version = "%s %s" % (version, svn_rev)
return version
I've looked at this file in emacs and I can't see the problem, and I've tried searching google with no luck. Can someone please point me in the right direction?
(For those interested in what I'm doing, please see http://www.allbuttonspressed.com/projects/djangoappengine.)
running this code in buffer with ipython shebang
! /usr/bin/env ipython
-- coding: utf-8 --
extending the end like this:
print version
return version
get_version()
get "1.3" at the ipython prompt
similar thing with python shell fails like this
from django.utils.version import get_svn_revision
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named utils.version
assume some install/path issue
Just an FYI, the latest version of django-appengine is found here:
https://github.com/django-nonrel
However, the allbuttonspressed version should be working. You may not have your python environment set properly.