I working with SQL Server 2019; I face issue I can't read data from Excel file using python from SQL Server.
Python used is version 3.10
What I try is:
EXECUTE sp_execute_external_script
#language = N'Python',
#script = N'import pandas as pd
df = pd.read_excel(r"\\192.168.7.9\Import\10\test\testData.xlsx", sheet_name = "Sheet1")
print(df)';
I get this error:
Msg 39019, Level 16, State 2, Line 48
An external script error occurred:
self.book = xlrd.open_workbook(self.io)
File "D:\SQL Data\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\xlrd_init.py", line 111, in open_workbook
with open(filename, "rb") as f:
PermissionError: [Errno 13] Permission denied: '\192.168.7.9\Import\10\test\testData.xlsx'
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "D:\SQL Data\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "D:\SQL Data\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 375, in rx_native_call
I check remote path I can exporter it and write to it and read.
I check Excel file it not open.
Related
I use WinSW for a Uvicorn service which basically provides an interface from which users can call certain SQL queries.
I have written a batch file which starts the program which works without a problem on manual execution with double clicking it. But if I run it as a service with WinSW and then start a query, there is output the error message:
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.
How can it be that the same .bat file works in standalone, but not as a service? And how do I fix it?
My service XML file with {Path_to_bat_directory} being a placeholder here in this post and being in real the hard coded directory path to the batch file containing the account name:
<configuration>
<id>X</id>
<name>X</name>
<description> XXX </description>
<executable>C:\Users\{Path_to_bat_directory}\StartUvicorn.bat</executable>
<onfailure delay="10 sec" action="restart"/>
<onfailure delay="20 sec" action="restart"/>
<onfailure action="none"/>
<resetfailure>1 hour</resetfailure>
<workingdirectory>C:\Users\{Path_to_bat_directory}</workingdirectory>
<priority>Normal</priority>
<stoptimeout>15 sec</stoptimeout>
<stopparentprocessfirst>true</stopparentprocessfirst>
<startmode>Automatic</startmode>
<waithint>15 sec</waithint>
<sleeptime>1 sec</sleeptime>
<logpath>C:\Users\{Path_to_bat_directory}\logs</logpath>
<log mode="roll"> </log>
</configuration>
The batch file StartUvicorn.bat:
call %~dp0win_env/Scripts/activate.bat
uvicorn main:app --port 8590 --host 0.0.0.0
The full error output as written into a log file is:
Exception in thread {X}:
Traceback (most recent call last):
File "{Path}\scoop\apps\anaconda3\current\lib\threading.py", line 973, in _bootstrap_inner
self.run()
File "{Path}\scoop\apps\anaconda3\current\lib\threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "{Path}\Sources\{Project}\.\populate_cache.py", line 25, in query_failures_from_db
df = pd.read_sql(statement, con, params=[date_from, date_to])
File "{Path}\Sources\{Project}\win_env\lib\site-packages\pandas\io\sql.py", line 563, in read_sql
pandas_sql = pandasSQL_builder(con)
File "{Path}\Sources\{Project}\win_env\lib\site-packages\pandas\io\sql.py", line 744, in pandasSQL_builder
import sqlite3
File "{Path}\scoop\apps\anaconda3\current\lib\sqlite3\__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "{Path}\scoop\apps\anaconda3\current\lib\sqlite3\dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.
Returns error below command using SQL Server Management Studio and SQL Server is running on Ubuntu 18.04.
EXEC sp_execute_external_script
#language = N'Python',
#script =
N'import pandas as pd
InputDataSet.to_csv("/home/dataexport/1.csv")',
#input_data_1 = N'select user_id from test.dbo.[users]'
GO
Error:
Msg 39004, Level 16, State 20, Line 11
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 11
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "/var/opt/mssql-extensibility/data/f131c9cc-88b2-4fb0-9611-311e79fe345b/EC1513EA-1DD0-4174-A820-4652B1F4EF6A/sqlindb_0.py", line 30, in transform
InputDataSet.to_csv("/home/dataexport/1.csv")
File "/opt/mssql/mlservices/runtime/python/lib/python3.7/site-packages/pandas/core/frame.py", line 1745, in to_csv
formatter.save()
File "/opt/mssql/mlservices/runtime/python/lib/python3.7/site-packages/pandas/io/formats/csvs.py", line 156, in save
compression=self.compression)
File "/opt/mssql/mlservices/runtime/python/lib/python3.7/site-packages/pandas/io/common.py", line 400, in _get_handle
f = open(path_or_buf, mode, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/home/dataexport/1.csv'
Msg 39019, Level 16, State 2, Line 11
An external script error occurred:
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "/opt/mssql/mlservices/libraries/PythonServer/revoscalepy/computecontext/RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "/opt/mssql/mlservices/libraries/PythonServer/revoscalepy/RxSerializable.py", line 375, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.
I have given full permissions on the SQL Server folders that are existing on /opt/ and /var/opt but no luck. even I gave Sudo permissions to users that are running SQL Server and LaunchPad service but didn't work. Even i given full permission on folder path /home/dataexport using chmod -R ugo+rwx /home/dataexport
I'm getting following error when I try to run the tcms-api module but following the steps given,
https://tcms-api.readthedocs.io/en/latest/modules/tcms_api.html#module-tcms_api
I'm using python 3 in CentOS, applied our own domain and certificates by mounting the certificates to docker container.
Can you please tell how to solve the SSL Certificate verification failure error?
[root#KiwiTCMS-Testcase-Portal docker-compose]# python3 test-api.py
Traceback (most recent call last):
File "test-api.py", line 5, in <module>
rpc_client = TCMS()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/tcms_api/__init__.py", line 123, in __init__
config['tcms']['url']).server
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/tcms_api/xmlrpc.py", line 124, in __init__
self.login(username, password, url)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/tcms_api/xmlrpc.py", line 131, in login
self.server.Auth.login(username, password)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__
return self.__send(self.__name, args)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request
verbose=self.__verbose
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request
return self.single_request(host, handler, request_body, verbose)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1166, in single_request
http_conn = self.send_request(host, handler, request_body, verbose)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1279, in send_request
self.send_content(connection, request_body)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/xmlrpc/client.py", line 1309, in send_content
connection.endheaders(request_body)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/http/client.py", line 1282, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/http/client.py", line 1042, in _send_output
self.send(msg)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/http/client.py", line 980, in send
self.connect()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/http/client.py", line 1448, in connect
server_hostname=server_hostname)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)
Introducing the following line fix the issue,
import ssl
try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
# Legacy Python that doesn't verify HTTPS certificates by default
pass
else:
# Handle target environment that doesn't support HTTPS verification
ssl._create_default_https_context = _create_unverified_https_context
I see that you are using Python 3.6 from RedHat's SoftwareCollections. That version contains a bug (or arguably a security feature) which doesn't respect settings documented in upstream Python which allow you to accept untrusted SSL certificates. There are lots of these things reported on bugzilla.redhat.com but I don't think they will change it!
This is how we do it in our test suite:
https://github.com/kiwitcms/tcms-api/blob/master/tests/krb5/integration_test.py#L18
Am getting an error in making migrations for my current django project.
below is the code written in settings.py file to connect to MSSQL Server.
DATABASES = {
'default': {
'ENGINE':'sql_server.pyodbc',
'NAME':'JTPROD',
'HOST':'TZACL5X8H1N2\SQLEXPRESS', ##this is my local machine database
'USER':'xxx', ##ommitted for the post
'PASSWORD':'xxx',##ommitted for tht post
'PORT':'',
'OPTIONS':{
'provider': 'SQLOLEDB', # Have also tried 'SQLCLI11' and 'SQLCLI10'
'extra_params': 'DataTypeCompatibility=80',
'driver':'SQL Server',
},
},
}
but am getting the following error when I run "python manage.py migrate
Traceback (most recent call last): File
"C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 216, in ensure_connection
self.connect() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 194, in connect
self.connection = self.get_new_connection(conn_params) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\sql_server\pyodbc\base.py",
line 307, in get_new_connection
timeout=timeout) pyodbc.OperationalError: ('08001', '[08001] [Microsoft][ODBC SQL Server Driver]Neither DSN nor SERVER keyword
supplied (0) (SQLDriverConnect); [08001] [Microsoft][ODBC SQL Server
Driver]Invalid connection string attribute (0)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "manage.py", line 15, in
execute_from_command_line(sys.argv) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management__init__.py",
line 381, in execute_from_command_line
utility.execute() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management__init__.py",
line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\base.py",
line 316, in run_from_argv
self.execute(*args, **cmd_options) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\base.py",
line 353, in execute
output = self.handle(*args, **options) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\base.py",
line 83, in wrapped
res = handle_func(*args, **kwargs) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\commands\migrate.py",
line 82, in handle
executor = MigrationExecutor(connection, self.migration_progress_callback) File
"C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\migrations\executor.py",
line 18, in init
self.loader = MigrationLoader(self.connection) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\migrations\loader.py",
line 49, in init
self.build_graph() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\migrations\loader.py",
line 212, in build_graph
self.applied_migrations = recorder.applied_migrations() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\migrations\recorder.py",
line 61, in applied_migrations
if self.has_table(): File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\migrations\recorder.py",
line 44, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File
"C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 255, in cursor
return self._cursor() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 232, in _cursor
self.ensure_connection() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 216, in ensure_connection
self.connect() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\utils.py",
line 89, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 216, in ensure_connection
self.connect() File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\db\backends\base\base.py",
line 194, in connect
self.connection = self.get_new_connection(conn_params) File "C:\Users\elukamis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\sql_server\pyodbc\base.py",
line 307, in get_new_connection
timeout=timeout) django.db.utils.OperationalError: ('08001', '[08001] [Microsoft][ODBC SQL Server Driver]Neither DSN nor SERVER
keyword supplied (0) (SQLDriverConnect); [08001] [Microsoft][ODBC SQL
Server Driver]Invalid connection string attribute (0)')
Does anyone have an idea on what I might be missing in my configuration ?
If you're still looking for an answer, I'm not sure this will help you, but it helped me when I was getting a similar error. SQL Server Express has TCP/IP connections disabled by default, but they apparently need to be enabled in order to allow a connection like this.
Open SQL Server Configuration Manager, expand 'SQL Server Network Configuration', and click on 'Protocols for SQLEXPRESS'. Assuming the entry for TCP/IP shows as Disabled, right-click that and select Properties. On the 'Protocol' tab, set Enabled to Yes. I also updated the 'IP Addresses' tab by entering '1433' into the TCP Port field of the 'IPAll' section (at the bottom). Then I specified 'PORT':'1433', in the settings.py DATABASES configuration section. I'm not sure that was necessary, but following those steps allowed my app to successfully connect to SQL Server Express.
I simply try to call
from moviepy.editor import VideoFileClip
but I receive this error
File "/Users/macbook/python/main_video.py", line 3, in <module>
from moviepy.editor import VideoFileClip
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/moviepy/editor.py", line 22, in <module>
from .video.io.VideoFileClip import VideoFileClip
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/moviepy/video/io/VideoFileClip.py", line 3, in <module>
from moviepy.video.VideoClip import VideoClip
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/moviepy/video/VideoClip.py", line 20, in <module>
from .io.ffmpeg_writer import ffmpeg_write_image, ffmpeg_write_video
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/moviepy/video/io/ffmpeg_writer.py", line 19, in <module>
from moviepy.config import get_setting
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/moviepy/config.py", line 38, in <module>
FFMPEG_BINARY = get_exe()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 86, in get_exe
raise NeedDownloadError('Need ffmpeg exe. '
imageio.core.fetching.NeedDownloadError: Need ffmpeg exe. You can download it by calling:
imageio.plugins.ffmpeg.download()
And if I try to call this one
imageio.plugins.ffmpeg.download()
Answer is
Imageio: 'ffmpeg.osx' was not found on your computer; downloading it now.
Error while fetching file: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>.
Error while fetching file: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>.
Error while fetching file: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>.
Error while fetching file: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>.
Traceback (most recent call last):
File "/Users/macbook/python/test.py", line 29, in <module>
imageio.plugins.ffmpeg.download()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 55, in download
get_remote_file('ffmpeg/' + FNAME_PER_PLATFORM[plat])
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/imageio/core/fetching.py", line 121, in get_remote_file
_fetch_file(url, filename)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/imageio/core/fetching.py", line 177, in _fetch_file
os.path.basename(file_name))
OSError: Unable to download 'ffmpeg.osx'. Perhaps there is a no internet connection? If there is, please report this problem.
What I can to do?
TRY
import imageio
imageio.plugins.ffmpeg.download()
Include above lines in your code . I had face same problem see the given picand this solved it.
Otherwise Check your internet connection.