When trying to use the hub.load function from tensorflow_hub, I get an OSError: SavedModel file does not exist at: error.
The weird thing is that it worked a few days ago, so I don't quite understand why I'm getting this error now.
Code to reproduce:
import tensorflow as tf
import tensorflow_hub as hub
URL = 'https://tfhub.dev/google/universal-sentence-encoder/4'
embed = hub.load(URL)
Specific error received:
OSError Traceback (most recent call last)
<ipython-input-11-dfb80f0299b2> in <module>
1 URL = 'https://tfhub.dev/google/universal-sentence-encoder/4'
----> 2 embed = hub.load(URL)
~/opt/anaconda3/lib/python3.7/site-packages/tensorflow_hub/module_v2.py in load(handle, tags)
100 if tags is None and is_hub_module_v1:
101 tags = []
--> 102 obj = tf_v1.saved_model.load_v2(module_path, tags=tags)
103 obj._is_hub_module_v1 = is_hub_module_v1 # pylint: disable=protected-access
104 return obj
~/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/saved_model/load.py in load(export_dir, tags)
576 ValueError: If `tags` don't match a MetaGraph in the SavedModel.
577 """
--> 578 return load_internal(export_dir, tags)
579
580
~/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/saved_model/load.py in load_internal(export_dir, tags, loader_cls)
586 tags = nest.flatten(tags)
587 saved_model_proto, debug_info = (
--> 588 loader_impl.parse_saved_model_with_debug_info(export_dir))
589
590 if (len(saved_model_proto.meta_graphs) == 1 and
~/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/saved_model/loader_impl.py in parse_saved_model_with_debug_info(export_dir)
54 parsed. Missing graph debug info file is fine.
55 """
---> 56 saved_model = _parse_saved_model(export_dir)
57
58 debug_info_path = os.path.join(
~/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/saved_model/loader_impl.py in parse_saved_model(export_dir)
111 (export_dir,
112 constants.SAVED_MODEL_FILENAME_PBTXT,
--> 113 constants.SAVED_MODEL_FILENAME_PB))
114
115
OSError: SavedModel file does not exist at: /var/folders/77/rvfl368x44s51r8dc3b6l2rh0000gn/T/tfhub_modules/063d866c06683311b44b4992fd46003be952409c/{saved_model.pbtxt|saved_model.pb}
So, just deleting that folder and running the hub.load() function again solves the issue
I have tried the above solution, but it didn't work for me...
Here's what worked:
Download the model from tfhub.dev with assets, variables and .pb checkpoint file.
Make sure you import tensorflow_text module!
import tensorflow_text
Specify the downloaded folder path in the hub.load() statement as in:
model =
hub.load("/Users/bilguun/Desktop/universal-sentence-encoder-multilingual-large_3/")
I was using i3d = hub.load(https://tfhub.dev/deepmind/i3d-kinetics-400/1).signatures['default'] as shown here
This method of loading the model worked in that day but after a few days I got the same error: OSError: SavedModel file does not exist at: C:\Users\catal\AppData\Local\Temp\tfhub_modules\092225fb776e28d6d64ac605ab6be03f18dd2027{saved_model.pbtxt|saved_model.pb}
After doing some research, I understood that the saved_model file location (specified in the error) was temporary so even if that folder still exists, there is no more saved_model.pb in it. So I downloaded the model linked here: https://tfhub.dev/deepmind/i3d-kinetics-400/1 and set i3d = hub.load("C:\\absolute_path_to_saved_model_folder").signatures['default'] and it worked.
So using Bilguun's answer really helped in my case.
This kind of error happens because the downloaded model is saved in temporary folder created by the application.
Since it is a temporary folder, the model saved in it may gets deleted or even the folder can be deleted.
When calling the hub.load() command, the programme checks for the temporary folder, if the folder is not found, the program will download the data from internet.
But if the folder is found and the data(or model) is not there, instead of downloading from the internet, it raises an error.
It can be resolved by deleting the temporary folder completely.
In Mac the temporary folder can be accessed by the command "open $TMPDIR" on the terminal( reference : https://osxdaily.com/2018/08/17/where-temp-folder-mac-access/).
The name of the folder can be get from the raised error and can be deleted easily .
Related
I am new to this, please tell me if the form is not correct.
I am trying to move my images from a file to a new empty files based on their name related to their key in the dictionary I created. Basically their label.
The dictionary is of this form:
print(multivalue_dict)
defaultdict(list,{'cat_a': ['55b85ea15a1536d46b7190ad6fff8ce7.jpg',
'd4684dcdc759dd9cdf41504698d737d8.jpg',
'6325b6870c54cd47be6ebfbffa620ec7.jpg'],
'cat_b': ['55b85ea15a1536d46b7190ad6fff8ce7.jpg',
'd4684dcdc759dd9cdf41504698d737d8.jpg',
'6325b6870c54cd47be6ebfbffa620ec7.jpg'],....}
The desired outcome would be then to have a file cat_a with all the images inside.
I am trying use the path and shutil
import os
import shutil
source_dir = "/Users/Me/Downloads/File/Images"
for key in multivalue_dict:
target_dir = f"/Users/Me/Downloads/File/{key}"
for file in multivalue_dict[key]:
shutil.move(os.path.join(source_dir, file_name), target_dir)
Though, I get the error:
Error Traceback (most recent call last)
/var/folders/gs/0bx6d0ms3jxfgjd6vfkz22l40000gn/T/ipykernel_29606/1312100907.py in
7 if os.path.exists(file):
8 pass
----> 9 shutil.move(os.path.join(source_dir, file_name), target_dir)
/Users/anaconda3/lib/python3.9/shutil.py in move(src, dst, copy_function)
821
822 if os.path.exists(real_dst):
--> 823 raise Error("Destination path '%s' already exists" % real_dst)
824 try:
825 os.rename(src, real_dst)
Error: Destination path '/Users/Me/Downloads/File/cat_a/f54d0c5cbd8e2270742293edf9075cb1.jpg' already exists
It seems that it is able to transfer 1 image to a new file but then stop, because it loops over it again.
Found the solution using iterrows() and pathlib rather than dictionary and shutil.
from pathlib import Path
for (index,row) in df.iterrows():
for label in label_list:
if row.product_cat == label:
p = Path(f'/Users/Me/Downloads/Images/{row.image}')
p.rename(Path(f'/Users/Me/Downloads/Images/{label}/{row.image}'))
I'm trying to build a blog using Hugo. Every time I run the hugo command, I get this error
ERROR 2021/06/22 18:57:50 JSON parse error: expected comma character
or an array or object ending on line 52 and column 40
12: {
^
Total in 145 ms
Error: Error building site: failed to render pages: JSON parse error:
expected comma character or an array or object ending on line 52 and column 40
12: {
^
I suspect my project directory structure to be the problem but I'm open to more questions about the situation.
My project directory structure looks like
root-folder
--content
--posts
--my-first-post.md
--posts-again
--my-second-post.md
--my-third-post.md
--my-fourth-post.md
and all markdown files contain exactly the following
---
title: "Project"
date: 2021-06-22T18:09:26+01:00
draft: false
---
This is a post
When draft is set to true, I don't get the error when I run the hugo command. The error comes up when draft for any of the files in posts or posts-again is set to false
I have searched every file for a line 52 that I can understand but I cannot find any.
Where could this error be coming from and how can I resolve it?
hugo serve and hugo serve -D work fine, but hugo doesn't
hugo-PaperMod is missing a comma in a JSON block in layouts/partials/templates/schema_json.html
I got this error and just pushed up a PR to fix it
https://github.com/adityatelange/hugo-PaperMod/pull/614
this may be your issue
Issue is baseurl set to "/" in config.toml.
Change it to "" and try
I'm using pydicom (installed with pip3, on python 3.7, using Idle) and I need to access pixel_array values.
I just copy-paste the example provided into the documentation and this leads to two errors:
first is about the get_testdata_files operation, which is not working because
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
>>>
==================== RESTART: D:\OneDrive\Desktop\test.py ====================
None
Traceback (most recent call last):
File "D:\OneDrive\Desktop\test.py", line 8, in <module>
filename = get_testdata_files("bmode.dcm")[0]
IndexError: list index out of range
I have solved this not using this operation.
second is about the pixel_array and I'm not so able to decode what is wrong, but it seems like the pixel_array is not populated. However I'm able to access other fields in the dataset and the file can be displayed (using ImageJ for example).
==================== RESTART: D:\OneDrive\Desktop\test.py ====================
None
Filename.........: bmode.dcm
Storage type.....: 1.2.840.10008.5.1.4.1.1.3.1
Patient's name...: Femoral trombenarterectomy, Case Report:
Patient id.......: Case Report 1
Modality.........: US
Study Date.......: 20110824
Image size.......: 768 x 1024, 27472108 bytes
Slice location...: (missing)
Traceback (most recent call last):
File "D:\OneDrive\Desktop\test.py", line 38, in <module>
plt.imshow(dataset.pixel_array, cmap=plt.cm.bone)
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\pydicom\dataset.py", line 949, in pixel_array
self.convert_pixel_data()
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\pydicom\dataset.py", line 895, in convert_pixel_data
raise last_exception
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\pydicom\dataset.py", line 863, in convert_pixel_data
arr = handler.get_pixeldata(self)
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\pydicom\pixel_data_handlers\pillow_handler.py", line 188, in get_pixeldata
UncompressedPixelData.extend(decompressed_image.tobytes())
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\PIL\Image.py", line 746, in tobytes
self.load()
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\PIL\ImageFile.py", line 261, in load
raise_ioerror(err_code)
File "C:\Users\marcl\AppData\Local\Programs\Python\Python37\lib\site-packages\PIL\ImageFile.py", line 58, in raise_ioerror
raise IOError(message + " when reading image file")
OSError: broken data stream when reading image file
Here is my code:
import matplotlib.pyplot as plt
import sys
import pydicom
import numpy
from pydicom.data import get_testdata_files
print(__doc__)
#filename = get_testdata_files("bmode.dcm")[0]
filename = "bmode.dcm"
dataset = pydicom.dcmread(filename)
# Normal mode:
print()
print("Filename.........:", filename)
print("Storage type.....:", dataset.SOPClassUID)
print()
pat_name = dataset.PatientName
display_name = pat_name.family_name + ", " + pat_name.given_name
print("Patient's name...:", display_name)
print("Patient id.......:", dataset.PatientID)
print("Modality.........:", dataset.Modality)
print("Study Date.......:", dataset.StudyDate)
if 'PixelData' in dataset:
rows = int(dataset.Rows)
cols = int(dataset.Columns)
print("Image size.......: {rows:d} x {cols:d}, {size:d} bytes".format(
rows=rows, cols=cols, size=len(dataset.PixelData)))
if 'PixelSpacing' in dataset:
print("Pixel spacing....:", dataset.PixelSpacing)
# use .get() if not sure the item exists, and want a default value if missing
print("Slice location...:", dataset.get('SliceLocation', "(missing)"))
# plot the image using matplotlib
plt.imshow(dataset.pixel_array, cmap=plt.cm.bone)
plt.show()
Could you help me to solve these two errors and access pixel_array values?
Don't hesitate to give me some advices /remarks/...
Thanks!
Hi Marc welcome to SO!
Your first error means that the get_testdata_files returns an empty list, so your file is not found. Have a look at the pydicom source, it shows that a search is performed in [DATA_ROOT]/test_files. Is your file located in that path?
Your second error is related to PIL and that can be quite difficult to debug and fix. First try to read the pixel_array from a dataset created from one of the supplied test files. If that works, your problem is probably that PIL cannot handle the specific encoding of your image data. You want to install and use GDCM instead of PIL to see if that solves the problem. Another user has had a similar issue as you, GDCM solved the problem. It can be a bit of a headache to get working unfortunately. Or have a look at this page, it shows some other alternatives on viewing the image data.
After transferring over a finished site to a new server, files/database and all, switching over the database settings in settings.php and running update.php I get the following errors:
Notice: Undefined index: name in block_menu() (line 146 of /home/nrsc2533/public_html/modules/block/block.module).
Notice: Undefined index: name in block_menu() (line 165 of /home/nrsc2533/public_html/modules/block/block.module).
Notice: Undefined index: name in system_menu() (line 647 of /home/nrsc2533/public_html/modules/system/system.module).
Notice: Undefined index: name in block_menu() (line 146 of /home/nrsc2533/public_html/modules/block/block.module).
Notice: Undefined index: name in block_menu() (line 165 of /home/nrsc2533/public_html/modules/block/block.module).
Notice: Undefined index: name in system_menu() (line 647 of /home/nrsc2533/public_html/modules/system/system.module).
I also lose most if not all of the items in the admin menu. I looked up the errors and they were all related to the following line of code:
'title' => check_plain($theme->info['name']),
This is a free theme that I customized, it is working fine on my test domain on my server, but not now that it has transferred. I did change the [theme].info file to reflect that it had been customized on the "name =" line. I checked online and found that the encoding may have been changed when I saved it, again strange since I had done this on the test domain and it seemed fine, so I changed the encoding to UTF-* without BOM and still to no avail. I have tried clearing the cache again via update.php and nothing changes. Any help will be GREATLY appreciated.
I'm getting this error... "ImportError: Could not find 'input_readers' on path 'map reduce'" when trying to Run my map reduce job via the http://localhost:8080/mapreduce launcher page.
It looks like my problem is similar to this post, AppEngine mapper API import error. Unfortunately, no definitive answers were given.
I've simplified it down to this tiny testmapreduce.py:
from google.appengine.ext import db
class TestEntity(db.Model):
value = db.StringProperty()
def mapperhandler(test):
print test.value
return
And my mapreduce.yaml:
mapreduce:
- name: Simplest MapReduce
mapper:
handler: testmapreduce.mapperhandler
input_reader: mapreduce.input_readers.DatastoreInputReader
params:
- name: entity_kind
default: testmapreduce.TestEntity
One possible clue is the presence of __init__.py has no effect (whether in the project root, the mapreduce directory, or both). I'm sure I'm making a beginner mistake, but over the last couple of days I have read every bit of documentation, and all the examples I can find. Thanks.
UPDATE:
I get the same error trying to invoke it via...
control.start_map(
"Give awards",
"testmapreduce.mapperhandler",
"mapreduce.input_readers.DatastoreInputReader",
{"entity_kind": "testmapreduce.TestEntity"},
shard_count=10)
UPDATE:
As requested, the stack trace -- let me know what else would be helpful...
ERROR 2011-10-16 17:09:27,216 _webapp25.py:464] Could not find 'input_readers' on path 'mapreduce'
Traceback (most recent call last):
File "/Develop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__
handler.get(*groups)
File "/Users/lc/PycharmProjects/mrtest/testmapreduce.py", line 22, in get
shard_count=10) # handle web form test interface
File "/Users/lc/PycharmProjects/mrtest/mapreduce/control.py", line 94, in start_map
transactional=transactional)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/handlers.py", line 811, in _start_map
mapper_input_reader_class = mapper_spec.input_reader_class()
File "/Users/lc/PycharmProjects/mrtest/mapreduce/model.py", line 393, in input_reader_class
return util.for_name(self.input_reader_spec)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 94, in for_name
module = for_name(module_name, recursive=True)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 102, in for_name
short_name, module_name))
ImportError: Could not find 'input_readers' on path 'mapreduce'
INFO 2011-10-16 22:09:27,253 dev_appserver.py:4247] "GET /giveawards HTTP/1.1" 500 -
This problem turned out to be that I was using the 2.7 version of the Python Interpreter in my local environment. When I switched to 2.5, it works fine.