I've created a classification model using AutoML Vision and tried to use this tutorial to make a small web app to make the model classify an image using the browser.
The code I'm using is basically the same as the tutorial with some slight changes:
<script src="https://unpkg.com/#tensorflow/tfjs"></script>
<script src="https://unpkg.com/#tensorflow/tfjs-automl"></script>
<img id="test" crossorigin="anonymous" src="101_SI_24_23-01-2019_iOS_1187.JPG">
<script>
async function run() {
const model = await tf.automl.loadImageClassification('model.json');
const image = document.getElementById('test');
const predictions = await model.classify(image);
console.log(predictions);
// Show the resulting object on the page.
const pre = document.createElement('pre');
pre.textContent = JSON.stringify(predictions, null, 2);
document.body.append(pre);
}
run();
This index.html file above is located in the same folder of the model files and the image file. The problem is that when I try to run the file I'm receiving this error:
error received
I have no idea what I should do to fix this error. I've tried many things without success, I've only changed the error.
models built with AutoML should not have dynamic ops, but seems that yours does.
if that is truly model designed using AutoML, then AutoML should be expanded to use asynchronous execution.
if model was your own (not AutoML), it would be a simple await model.executeAsync() instead of model.execute(), but in AutoML case, that part is hidden inside AutoML library module inside classify call and that needs to be addressed by tfjs AutoML team.
best to open an issue at https://github.com/tensorflow/tfjs/issues
btw, why post a link to an image containing error message instead of copying the message as text here??
Related
I have a tensorflow.js script/app that runs in Node.js using tfjs-node and Universal Sentence Encoder (USE).
Each Time the script runs, it downloads a 525 MegaByte File (the USE model file).
Is there any way to load the Universal Sentence Encoder Model File from the local file system to avoid downloading such a large file every time I need to run the node.js tensorflow script?
I've noted several similar model loading examples but none that work with Universal Sentence Encoder as it does not appear to have the same type functionality. Below is a stripped down example of a functioning script that downloads the 525 MB file every time it executes.
Any help or recommendations would be appreciated.
const tf = require('#tensorflow/tfjs-node');
const use = require('#tensorflow-models/universal-sentence-encoder');
// No Form of Universal Sentence Encoder loader appears to be present
let model = tf.loadGraphModel('file:///Users/ray/Documents/tf_js_model_save_load2/models/model.json');
use.load().then(model => {
const sentences = [
'Hello.',
'How are you?'
];
model.embed(sentences).then(embeddings => {
embeddings.print(true /* verbose */);
});
});
I've tried several recommendations that appear to work for other models but not Universal Sentence Encoder such as:
const tf = require('#tensorflow/tfjs');
const tfnode = require('#tensorflow/tfjs-node');
async function loadModel(){
const handler = tfnode.io.fileSystem('tfjs_model/model.json');
const model = await tf.loadLayersModel(handler);
console.log("Model loaded")
}
loadModel();
its not a model issue per-say, its a module issue.
model can be loaded any way you want, but the module #tensorflow-models/universal-sentence-encoder implements only a specific internal way on how it loads actual model data.
specifically, it internally uses tf.util.fetch.
solution? use some library (or write your own) to register a global fetch handler that knows how to handle file:// prefixes - if global fetch handler exists, tf.util.fetch will simply just use it.
hint: https://gist.github.com/joshua-gould/58e1b114a67127273eef239ec0af8989
I want to compress/encode gltf file using draco programatically in threejs with reactjs. I dont want to use any commandline tool, I want it to be done programatically. Please suggest me a solution.
I tried using gltf-pipeline but its not working in client side. Cesium library was showing error when I used it in reactjs.
Yes, this is can be implemented with glTF-Transform. There's also an open feature request on three.js, not yet implemented.
First you'll need to download the Draco encoder/decoder libraries (the versions currently published to NPM do not work client side), host them in a folder, and then load them as global script tags. There should be six files, and two script tags (which will load the remaining files).
Files:
draco_decoder.js
draco_decoder.wasm
draco_wasm_wrapper.js
draco_encoder.js
draco_encoder.wasm
draco_encoder_wrapper.js
<script src="assets/draco_encoder.js"></script>
<script src="assets/draco_decoder.js"></script>
Then you'll need to write code to load a GLB file, apply compression, and do something with the compressed result. This will require first installing the two packages shown below, and then bundling the web application with your tool of choice (I used https://www.snowpack.dev/ here).
import { WebIO } from '#gltf-transform/core';
import { DracoMeshCompression } from '#gltf-transform/extensions';
const io = new WebIO()
.registerExtensions([DracoMeshCompression])
.registerDependencies({
'draco3d.encoder': await new DracoEncoderModule(),
'draco3d.decoder': await new DracoDecoderModule(),
});
// Load an uncompressed GLB file.
const document = await io.read('./assets/Duck.glb');
// Configure compression settings.
document.createExtension(DracoMeshCompression)
.setRequired(true)
.setEncoderOptions({
method: DracoMeshCompression.EncoderMethod.EDGEBREAKER,
encodeSpeed: 5,
decodeSpeed: 5,
});
// Create compressed GLB, in an ArrayBuffer.
const arrayBuffer = io.writeBinary(document); // ArrayBuffer
In the latest version of 1.5.0, what are these two files? draco_decoder_gltf.js and draco_encoder_gltf.js. Does this means we no longer need the draco_encoder and draco_decoder files? and how do we invoke the transcoder interface without using MeshBuilder. A simpler API would be musth better.
I am uploading user profile image which is uploading and moved to storage/app/upload/images folder but when I am trying to display that image, below given error occurs.
Method App\Image::__toString() must not throw an exception, caught Illuminate\Database\Eloquent\JsonEncodingException
Here is my controller function for displaying
public function userProfile() {
$image = Image::all();
return view('frontend.layouts.Profile',compact('image'));
}
My view in which I am displaying image
#foreach($image as $images)
<img style="width:210px ; height: 230px " src="/storage/app/upload/images/{{$images->image}}" >
#endforeach
Please Upload your image in Public directory and then try to access that, it will work fine
There are three ways of making an image available to a user:
1. As a public asset
Here the image is made available to everyone. For instance your website logo, or landing page image would be accessed by all. So there is a url to the image that is easily accessed by all. These sort of files would go straight to public/img/ folder.
2. As a protected image available only if specific url is requested
Here user specific images would be accessed by specific people. Think of your members' personal photos that you want to make available only to the member herself or to some specific person. In this case you would store the images in storage/app/public and make a symlink using the artisan command php artisan storage:link You can read more on this here. Assuming that you store your files using random names using str_random() you would then generate urls to your image using the asset() helper like: echo asset('storage/X3jf5j5b2j3n.jpg'); Given that the file names are random, it would be hard to access this image by everyone excepting those who have the url generated using the asset() helper.
3. As a protected image made available using Intervention library
In this case you would first check if user is logged in and then dynamically load the image using Intervention via another protected route. So in your web routes you would first have the user authorization using auth middleware:
Route::group(['middleware' => 'auth'], function () {
Route::get('user', 'UserController#userProfile');
Route::get('images/{image}', 'UserController#serveImage'); // this route serves the image only if user is logged in
});
Then once your have installed Intervention library using composer, our UserController would look like:
use Intervention;
class UserController extends Controller
{
public function userProfile()
{
$images = Image::all();
return view('frontend.layouts.Profile', compact('images'));
}
public function serveImage($image)
{
$filename = storage_path('app/images/'.$image);
return Intervention::make($filename)->response();
}
}
You can see that the image is now being served from the storage folder and not public folder. So this method serveImage() is called only when the route defined earlier for it is authorized. Intervention then works its magic to read the image and send it as a http response.
Your view would change one tad bit to accommodate the new route end point that we defined called images. I assume here that you are storing the filename of the image in db by a field named filename:
#foreach($images as $image)
<img style="width:210px ; height: 230px " src="{{ url('/images/'.$image->filename) }}" >
#endforeach
Note: Do bear in mind that the preferred way to serve images is by using method 2 since it is much faster. You can use method 3 sparingly if you really don't want anyone to even stumble upon the files using the urls.
I am trying to build a simple photo upload app on Ionic (Cordova). I am using the cordovaImagePicker plugin to have the user select images from the mobile device. This plugin returns an array of paths on the device.
For handling the upload part I am using jquery-file-upload (mostly because that is what I used for the browser version and I am doing all kinds of processing for which I have the code ready). The problem is however that jquery-file-upload expects to work with an input element <input type="file"> which creates a javascript File object containing all kinds of metadata.
So in order to get the cordovaImagePicker to work with jquery-file-upload, I figure I have to convert the filepath to a File object. Below I am using the cordova file plugin to achieve this:
$cordovaImagePicker.getPictures($scope.pickOptions).then(function(filelist) {
$.each(filelist, function (index, filepath) {
$window.resolveLocalFileSystemURL(filepath, function(fileEntry) {
fileEntry.file(function(file) {
var reader = new FileReader();
reader.onloadend = function(e) {
fileObj = new File([this.result],"filename.jpg",{type: "image/jpeg"});
// send filelist from cordovaImagePicker to jquery-fileupload as if through file input
$('#fileupload').fileupload('send', {files: fileObj});
};
reader.readAsArrayBuffer(file);
}, function(e){$scope.errorHandler(e)});
}, function(e){$scope.errorHandler(e)});
});
}, function(error) {
// error getting photos
console.log('Error selecting images through $cordovaImagePicker');
});
So first of all this is not really working correctly, apparently I am doing doing something wrong, since for example the type attribute ends up being another object that contains the type attribute with the correct values (and other such weird issues). I would be happy if someone could point out what I am doing wrong.
Surely there must be something (cordova plugin?) that I am not aware of that does this conversion for me (including for example adding a thumbnail)? Alternatively, maybe there is something that can easily make jquery-file-upload work with filepaths? I couldn't find anything so far...
However, it feels I am trying too hard here to force connecting two components that were just not built to work together (File objects vs filepath) and I should maybe just rewrite the processing and use the cordova file transfer plugin?
I ended up rewriting the uploader with the cordova-file-transfer which works like a charm. I wasted more time trying to work around it than just rewriting it from scratch.
I want to parse Swagger data from the JSON I get from {service}/swagger/docs/v1 into dynamically generated .NET class.
The problem I am facing is that different APIs can have different number of parameters and operations. How do I dynamically parse Swagger JSON data for different services?
My end result should be list of all APIs and it's operations in a variable on which I can perform search easily.
Did you ever find an answer for this? Today I wanted to do the same thing, so I used the AutoRest open source project from MSFT, https://github.com/Azure/autorest. While it looks like it's designed for generating client code (code to consume the API documented by your swagger document), at some point on the way producing this code it had to of done exactly what you asked in your question - parse the Swagger file and understand the operations, inputs and outputs the API supports.
In fact we can get at this information - AutoRest publically exposes this information.
So use nuget to install AutoRest. Then add a reference to AutoRest.core and AutoRest.Model.Swagger. So far I've just simply gone for:
using Microsoft.Rest.Generator;
using Microsoft.Rest.Generator.Utilities;
using System.IO;
...
var settings = new Settings();
settings.Modeler = "Swagger";
var mfs = new MemoryFileSystem();
mfs.WriteFile("AutoRest.json", File.ReadAllText("AutoRest.json"));
mfs.WriteFile("Swagger.json", File.ReadAllText("Swagger.json"));
settings.FileSystem = mfs;
var b = System.IO.File.Exists("AutoRest.json");
settings.Input = "Swagger.json";
Modeler modeler = Microsoft.Rest.Generator.Extensibility.ExtensionsLoader.GetModeler(settings);
Microsoft.Rest.Generator.ClientModel.ServiceClient serviceClient;
try
{
serviceClient = modeler.Build();
}
catch (Exception exception)
{
throw new Exception(String.Format("Something nasty hit the fan: {0}", exception.Message));
}
The swagger document you want to parse is called Swagger.json and is in your bin directory. The AutoRest.json file you can grab from their GitHub (https://github.com/Azure/autorest/tree/master/AutoRest/AutoRest.Core.Tests/Resource). I'm not 100% sure how it's used, but it seems it's needed to inform the tool about what is supports. Both JSON files need to be in your bin.
The serviceClient object is what you want. It will contain information about the methods, model types, method groups
Let me know if this works. You can try it with their resource files. I used their ExtensionLoaderTests for reference when I was playing around(https://github.com/Azure/autorest/blob/master/AutoRest/AutoRest.Core.Tests/ExtensionsLoaderTests.cs).
(Also thank you to the Denis, an author of AutoRest)
If still a question you can use Swagger Parser library:
https://github.com/swagger-api/swagger-parser
as simple as:
// parse a swagger description from the petstore and get the result
SwaggerParseResult result = new OpenAPIParser().readLocation("https://petstore3.swagger.io/api/v3/openapi.json", null, null);