I am developing a google smart home action app for interacting with our security systems. This includes also controlling relays for switches, outlets, lights, as well as thermostats for heating or cooling systems and some other peripherals, besides the original arming/disarming capabilities of the system.
Is there a place where I can find a sort of utterance reference (at least in English or in any Western Europe language)?
My company is Italian (as am I), and I am struggling to test the action (which is currently in DRAFT): most of the commands must be inferred by examples on the types & traits official documentation.
But this is far from being complete, plus it's in English only. Most of the times I'm trying to invoke the action using an Italian guessed translation from English examples but it's not working or understood by the assistant.
I also need to create a user manual for our customers in the available languages (I currently speak Italian and English only), and I need to give examples to the translators.
I cannot find anything anywhere.
Is there anything of the sort? Even a partial/incomplete list would do, to begin with.
Our internal team is constantly working on updating the public documentation for Smart Home devices with sample utterances for controlling and interacting with them in the list of languages supported by the traits.
Right now we have provided utterances for devices in en-US, de-DE and fr-FR. More information about the sample utterances for Window (as an example) device type can be found here
https://developers.google.com/assistant/smarthome/guides/window#fr-fr
Meanwhile you can use Google Translate to convert phrases to other languages, and see if the translations are working for you.
Related
I have been trying for a while now to create a kind of "controller" for my windows pc which is integrated with google assistant.
I would like to have actions like "Turn off the internet", "Shutdown" or "Open program".
However in trying to implement this with Google smart action API I see that it "only" supports a set of traits. I know I can "reuse" different traits for different actions, like "OnOff" to control pc power, but how about starting programs on my pc?
I would like to use natural commands (Hey Google, open Word), instead of the conversational API which is (Hey Google, talk to my pc -> open Word), but I would understand if that is not possible, since the traits are how Google HomeGraph knows which devices supports the users query.
Am I going into a rabbithole of impossibilities and should I just bite the bullet and use the Conversational API?
Any open source projects which is doing this would also be appreciated, end goal is implementing something myself, but inspiration is always nice.
I previously built my own PC integration with the smart home platform and gave it a few commands.
Going through smart home means that you cannot add custom device traits. Whichever exist in the platform are it. However, there are quite a few traits available for use.
In your set of use cases, there is an AppSelector trait and a NetworkControl trait. You may be interested in using both of these.
Whats the best approach of using existing NLP tools in english with another language ex.spanish ?
That's an awfully broad question, and you'd need to provide some more pointers. However, if you're interested in general research on the topic, you can try Hana, Feldman, Brew (2004) "Tagging Russian using Czech morphology" and Resnik's 2004 "Using bilingual text for monolingual annotation" and start from there.
In general, you'd want to have a bicorpus (say, English/Swedish). Then establish mappings using alignment (that's a common topic in machine translation with many established results.)
You can then tag the English side, and use the mapping to "translate" these mappings into the Swedish side. Then you can train the same tool that created the mappings on the English side using the newly annotated Swedish corpus.
It goes without saying that you'll lose quite a bit of quality and that this technique only works for supervised methods. You should probably try to find properly annotated Swedish corpora and tools. There are a few out there.
Im starting a new project. The aim of the project is to create a e-authoring tool for building courses in SCORM Complaint. Im new to this domain and I have little idea on this. I have taken a view on authoring tool in Articulate, which my customer requires to do the same. I understood the content creation, but I am trying to understand How can I export this as SCORM compliant course? In between I learned about xAPI as well And understood it is a kind of enhanced SCORM.
Could any one guide me to understand this,
1) How can create content from my custom authoring tool and export as SCORM complaint
2) Is it better to use xAPI or SCORM.
3) How is the SCORM pacakge communicate with my custom made LMS?
4) Heard about LRS,
My custom authoring tool will be made in React and store would MondDB
Any help would be greatly appreciated. Thankyou!
That is a lot to take on, particularly all at once.
1) The SCORM spec is made up of multiple parts. There is a packaging portion and a runtime portion. The basics are that your package needs to be a zip file, and that zip needs to include specific files that indicate to the LMS what type of standard it is along with other metadata about the package. For SCORM this will be called an imsmanifest.xml file. For xAPI you are most likely going to use a cmi5.xml (see cmi5) or a tincan.xml file (what Articulate Storyline exports when it says "xAPI"). The other parts of the package will depend on what standard and version of that standard (for SCORM 1.2, 2004 2nd, 3rd, or 4th edition) you are targeting, realizing that different LMSs support different standards and different degrees of those standards.
Once you have a package constructed that will import, the content itself (usually an HTML file) will need to locate the JavaScript API provided by the SCORM player (from the LMS) and make specific calls depending on what the content is needing to store or read, this is the runtime portion. The calls will again depend on the standard and version. For xAPI based packages (either tincan.xml packages or cmi5 packages) the content will communicate directly to the LRS based on the information provided on the URL at launch time (there is no built in JavaScript API).
2) This entirely depends on what your customer base looks like and the types of data that you intend to capture. SCORM is a more mature landscape and has wider adoption and is more heavily specified, if the information you need to capture fits into its limited information model then it is still an excellent choice. If you need significant data portability and/or the information you need to capture goes beyond compliance data (pass/fail, complete, and score) and/or interaction data (questions + answers) then you should consider xAPI, specifically via cmi5.
3) The LMS must provide a JavaScript API (specified by the SCORM runtime) which the content will use as its interface. The storage/retrieval of data is implementation specific for the LMS beyond what is included in the specification for the JavaScript API.
4) You didn't really include a question here.
I would suggest familiarizing yourself with the two sets of standards via http://scorm.com and http://xapi.com. And although it is a plug for my company's product, you may want to consider the Rustici Driver as it is a product (library) specifically designed to make it easy for an authoring tool to export content as SCORM 1.2, 2004, AICC, cmi5 or Tin Can (the latter two being xAPI). Once you have your tool up and running with minimal standards support you should consider testing it on Rustici's SCORM Cloud (it is free for this purpose), see http://cloud.scorm.com.
The format is huge, there is no quick reference guides. And different authoring tools have different scorm-support depths. You should probably start with this document
Sounds like you're talking about designing editable content; and the content "framework" itself.
This is a massive effort! This is massive support! That said, people do it.
Having built a CMS system for many supporting subject matters I had to divide and conquer this task.
Few ways I'd think to digest this beast- data, data, data
Requirements on Activities (Interaction types)
Design (static/dynamic) on these interactions
The view/facade displaying can change. Tech moves at the speed of light. Need to come up with a super solid data model.
I'd think about how these can be generic, and how they can be extended to meet the customers goals/needs. All depends how much customization (if any) can happen.
I start mapping all this to SCORM CMI Object level calls. Scoring, Progress, Interactions, Objectives etc...
Get your self a wicked SCORM Content API library or write one yourself. You'll be re-using a lot of these calls, no sense baking them into all your interactions
Get up on SCORM Packaging .. much of this has to be defined at author time. Lots of reading, and a lot of features you need to pick thru if your customers even use. Don't dev in places that have .1% market need. The low hanging fruit get you to market.
Surround yourself with passionate great people. You'll need them.
As far as the standards go, it's all about portability. SCORM works directly with a LMS if thats where your customer goes. Others use a LRS which is coded to work with one they set at author time. You can even do both.
Aside from React and MongoDB, you'll need something that can do the lift and shift of all this content.
I am studying about artificial intelligence
How did the virtual assistants analyze questions?
Example: when I said "Way from New York City to Washington DC" , va opened google map?
If I would develop that stuff myself I'd probably would use pattern like
If ({a} {linker} {b}) where a and b are in list of cities on map or something related to that and linker is word like "from" or "to" i would run google maps with, also if any of remaining words would signal that it is map related stuff, like "location", "map", "route" and so on...
Just a guess but I think this should work like that
I don't have any idea that which programming language will you prefer to develop this project but I highly recommend Python. It is object-oriented, high level and extensive support libraries. Moreover, I don't know what is your targeted OS for this project, you unfortunately have not specified it and maybe you will choose Android OS(for mobile application you may use Python) so Python may not be a good option for you. But I supposed that you will develop a desktop application so I will propose my opinions under the motivation that you will choose Python to develop this project.
First of all, you can use speech recogition library for speech to text. After getting text from speech, we can jump to next step which is analyzing the questions.
At the present time, deep learning is the state of the art and Tensorflow is the great technology to take advantage of deep learning.
This is an amazing chatbot framework to build a conversational model for your custom chat bot. You should edit this JSON file to create your conversational intents, after the editing JSON file, you can analyze the questions of user (i.e. your program can understand what user said, the program will parse the question and get the location which is requested by user). When you get the location from question, the program can open browser (for example GoogleMaps) via executing an Ubuntu terminal command in Python.
As a summary, when user says "Way from New York City to Washington DC" the program will run respectively;
Get text from user speech
The program will analyze text via trained system and it can understand that what user said
The program can get the destination and current location which is specified by user (many kind of information can get from user request) by using the structure of this JSON[I recommend this way], or may be NLP or using any kind of string operations
The program will start to load Google maps URL via (for example) Ubuntu terminal command for these location information
Is it possible to launch an Alexa App with just its name? This is similar to when you ask it what the weather is.
"Alexa, weather"
However I would like to be able to say
"Alex, weather in Chicago" and have it return that value
I can't seem to get the app to launch without a connecting word. Things like ask, open, tell would count as a connecting word.
I have searched the documentation but can't find mention of it, however there are apps in the app store that do this.
It is documented in the first item here.
I've verified that this works with my own skill. One thing I've noticed is that Alexa's speech recognition is much worse when invoked in this manner presumably because it requires matching against a greater set of possible words. I have to really enunciate in a quiet room to get Alexa to recognize my invocation name in this context.
When developing a custom skill you have to use the connecting words e.g. Alexa, ask your invocation name to do something.
If you want to pass a variable, you have to specify the sample utterances:
OneshotTideIntent get high tide
OneshotTideIntent get high tide for {City} {State}
Then you handle cases in your code when user does not provide these values. For examples see https://github.com/amzn/alexa-skills-kit-js
When writing the example phrases you use the following construct:
"Alexa, [connecting word] [your invocation name], [sample utterance]". As far as I have noticed she is rather picky and you have to be exact when invoking custom skill (the voice recognition works way better with built in skills)
EDIT: launching skill without connecting word is possible when developing "smart home" skill