How to add custom span into Trace, in go - google-app-engine

My app runs on App Engine Standard and the Go runtime.
I have this trace for my recent request:
There is a big gap between the "urlfetch" span and the "datastore_v3" span, because my app processes some CPU-bound computation for ~1000ms.
I would love to programmatically add my computation as a custom span into the Trace view, and get something like this:
Is there a way to do this in my app written in go? (source here)

It appears it might be possible. From Setting Up Stackdriver Trace for Go:
Alpha
This is an alpha release of the OpenCensus package for Go. These
libraries might be changed in backward-incompatible ways and are not
recommended for production use. They are not subject to any SLA or
deprecation policy.
Stackdriver Trace can be used by Go applications using the
OpenCensus package for Go.
Stackdriver Trace's Go support is provided by OpenCensus, a set
of tracing and application metrics instrumentation libraries that work
with multiple backends. The latest details about OpenCensus for Go,
along with additional documentation and examples, can be found on its
GitHub page.
Support is enabled by default in the flexible environment, however the docs make no mention about the standard environment (if that's your case I'd say just give it a try). From App Engine:
On Google App Engine flexible environment, the Stackdriver Trace API
access scope is enabled by default, and the OpenCensus client
library can be used without needing to provide credentials or a
project ID.
An application code sample is provided on the same page.

I could make it work with the new AppEngine runtime for Go 1.11 (currently in beta) and OpenCensus with the Stackdriver exporter.
In order to attach my custom span to the main Trace of the request, I use this utility func:
// Start a new span "With Remote Parent"
func startSpanfWRT(r *http.Request, msg string, args ...interface{}) (c2 context.Context, endSpan func()) {
caption := fmt.Sprintf(msg, args...)
c := r.Context()
spanContext, ok := (&propagation.HTTPFormat{}).SpanContextFromRequest(r)
if !ok {
return c, func() {}
}
var span *trace.Span
c2, span = trace.StartSpanWithRemoteParent(c, caption, spanContext)
endSpan = func() {
span.End()
}
return c2, endSpan
}
Note that it requires the *http.Request as argument (a context.Context wouldn't be enough here).
Here is the sample app source code.
As a span needs to be started and then later stopped, the start func returns an "end" callback, and a new Context as well.
It is fine to call startSpanfWRT multiple times, and they may overlap. It requires to pass the *http.Request around, which is not super-convenient (usually we only pass Contexts around).
However, after a call to startSpanfWRT, you may add children spans conveniently, just paying attention to the respective Contexts:
c2, childSpan := trace.StartSpan(c, caption)

Related

Flink Stateful functions - How to register statefun's ValueSpec in embedded module

I am refactoring my Flink Statefuns for embedded service deployment, but I cannot find the way to register statefun's ValueSpec in EmbeddedModule
When I was using remote http service deployment on k8s, I was building StatefulFunctionSpec with statefun's all value specs inside, and was registering it by StatefulFunctions.withStatefulFunction(spec):
Build static spec in MyStatefun:
static StatefulFunctionSpec SPEC = StatefulFunctionSpec.builder(MY_TYPE)
.withValueSpecs(MY_VALUE_SPEC)
.withSupplier(MyStatefun::new)
.build();
Register it in StatefulFunctions:
StatefulFunctions functions = new StatefulFunctions();
functions.withStatefulFunction(MyStatefun.SPEC);
And finally use functions.requestReplyHandler() as a handler in http server.
What's the way of doing that in EmbeddedModule?
In examples that I found so far, I only see that statefun is registered as this, but does this also register ValueSpecs?
binder.bindFunctionProvider(MyStatefun.FUNCTION_TYPE, x -> (StatefulFunction) new MyStatefun());
UPD. It seems that .bindFunctionProvider(...) cannot bind sdk.java.StatefulFunction, it can only bind sdk.StatefulFunction, which implements .invoke(sdk.Context, Object o) instead of .apply(sdk.java.Context, Message msg). And sdk.Context does not have .storage() method for accessing ValueSpecs.
UPD2. Found page about PersistedValue that could be used for state management and could possibly be the answer to my question: https://nightlies.apache.org/flink/flink-statefun-docs-master/docs/sdk/flink-datastream/

How do I add custom tracing to a second-generation App Engine application in Go?

Google App Engine now supports Go 1.11 via the new second-generation standard environment. While converting an older standard environment application to the second generation, it wasn't obvious how to combine tracing information from the app engine infrastructure with custom tracing I added to the application using OpenCensus.
Even though I had created a stackdriver exporter and registered traces, I wasn't seeing custom trace information in the stackdriver console attached to inbound requests.
The key is I was missing is understanding how span context is communicated to the serving app. Google leverages the X-Cloud-Trace-Context header to propagate span context within requests sent to your serving instances, and the go.opencensus.io/exporter/stackdriver/propagation library provides an implementation to extract and persist this information within http requests.
Don't forget to create a stackdriver exporter, and register traces to it. The docs for the exporter library show an example of this.
// CreateSpanFromRequest returns a context and span based on the http.Request.
// If no existing spancontext is found, this will start a new span.
// Modifies existing request to contain the generated span's context.
func CreateSpanFromRequest(name string, r *http.Request) (context.Context, *trace.Span) {
var span *trace.Span
ctx := r.Context()
httpFormat := &propagation.HTTPFormat{}
sc, ok := httpFormat.SpanContextFromRequest(r)
if ok {
ctx, span = trace.StartSpanWithRemoteParent(ctx, name, sc)
} else {
ctx, span = trace.StartSpan(ctx, name)
}
// Write the span context into the http.Request. We do this to
// to enable chaining handlers together more easily.
httpFormat.SpanContextToRequest(span.SpanContext(), r)
return ctx, span
}
Using this, I was able to add custom spans to my handlers that would be properly associated with the incoming request information in stackdriver:
func indexHandler(w http.ResponseWriter, r *http.Request) {
_, span := CreateSpanFromRequest("indexHandler", r)
defer span.End()
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
fmt.Fprint(w, "Hello, World!")
}

Application getting crashed on setting the vocabulary for carName using Siri Kit

I am trying to develop an application using SiriKit to get the car door lock status and set the same from Siri. I followed this blog https://www.appcoda.com/sirikit-introduction/ and did all the setup replacing the INStartWorkoutIntent with INGetCarLockStatusIntent.
But when i try to set the vocabulary for carName, the application is getting crashed with following exception,
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Illegal attempt to provide vocabulary of type INVocabularyStringTypeCarName by an app that does not handle any intents that could use that type of vocabulary'
The source code that i am using to set the vocabulary is,
INPreferences.requestSiriAuthorization { (status) in
}
INVocabulary.shared().setVocabularyStrings(["benz", "bmw", "audi"], of: .carName)
In AppDelegate, i have added the following method,
func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: #escaping ([Any]?) -> Void) -> Bool {
guard let intent = userActivity.interaction?.intent as? INGetCarLockStatusIntent else {
print("AppDelegate: Start Workout Intent - FALSE")
return false
}
print("AppDelegate: Start Workout Intent - TRUE")
print(intent)
return true
}
Also created the extension for intent handler and implemented INSetCarLockStatusIntentHandling, INGetCarLockStatusIntentHandling protocols. I am getting this issue when i try to run it in iPhone 10.
Check, if in TARGETS of your project in Build Phases->Embed App Extensions added your Siri Extension. Maybe if you replace the INStartWorkoutIntent with INGetCarLockStatusIntent, old INStartWorkoutIntent remained there.
My crash fix this.
I was facing a similar issue. Make sure your extension's Deployment Target is set to appropriate iOS version. Creating an extension with the latest Xcode (at the moment 10.1) will set the Deployment Target to 12.1 and thus cause crash when run on iOS 10. So you should change it to your desired minimum.

Go App Engine get version in init() without Context

Is there a way to to get my autoscaled application's VersionID in my init() function without a Context? The only available option seems to be appengine.VersionID(context.Context). Manually scaled instances have /_ah/start called when they start up (giving access to a Context), but there is nothing like this for autoscaled instances.
I am not caring about the generated ID that appengine.VersionID returns with it, just the app.yaml version.
EDIT: A bit of context: I am wanting to deploy versions in the form x-x-x-dev or x-x-x-live and have my database connection depend on the version suffix. This way, when I look in the GCP console, I can be certain which deployed modules/services are using which database. Of course, I setup my DB connection pool in the init(), which has no access to a Context.
I searched and searched with no answers online anywhere, so here it is.
Simply parse the app.yaml file in your init() function. My example here uses a yaml parsing package, but it can be done more lightweight if you need.
import "github.com/ghodss/yaml"
type AppVersion struct {
Version string `json:"version"`
}
func VersionID() (string, error) {
dat, err := ioutil.ReadFile("app.yaml")
if err != nil {
return "", err
}
a := &AppVersion{}
err = yaml.Unmarshal(dat, a)
if err != nil {
return "", err
}
return a.Version, nil
}
Note that this DOES NOT return the generated ID in the form X.Y that appengine.VersionID() does. Only the X part of the version.
As an aside, in the appengine repo on Github, the actual call to appengine.VersionID requires a Context, but internally calls the internal package with nil. So they basically force you to call it with a Context, but it isn't actually used. It's incredibly infuriating.
EDIT: It should be noted that the new Go SDK in gcloud no longer supports version in the app.yaml, as it is now a required parameter at deploy. However, the "legacy" SDK is still supported and maintained, which I am continuing to use as of today (12/24/2018).

How to register own protocol using the WebBrowser control?

In a WP7 Silverlight application with a WebBrowser control I want to use an own protocol like "myttp://" to deliver some local content. I can't use Navigate() to an IsolatedStrorage because some content will by created on demand. For the same reason NavigateToString() is also not usable for me.
I tried to register a WebRequestCreator descend for my MYTP protocol
myCreator = new MyRequestCreator();
WebRequest.RegisterPrefix("mytp://", myCreator);
but it isn't called from the browser control if I navigate to "mytp://test.html".
If I create a WebRequest via code
WebRequest request;
request = WebRequest.Create("mytp://test.html");`
everythings works fine.
Any suggestions what is wrong or how to do it?
The WebBrowser control will use the Windows Phone Internet Explorer Browser's HTTP stack to statisfy web requests. This HTTP stack is entirely separate from the Client HTTP stack being used by the application. Hence the browser does not see your protocol at all.
I agree with AnthonyWJones words, though I dont know, what exactly he meant by "Browser HTTP stack".
The standard Silverlight's "access to Browser's stack" (used to handle sessions etc) in form of System.Net.Browser.WebRequestCreator.BrowserHttp httprequest factory (versus the "normal/aside" System.Net.Browser.WebRequestCreator.ClientHttp factory) is actually available to the application code in WP7. It is hidden from the SDK, but available on the device and with small effort, the application can use it, for example, to have its emitted cookies in sync with the Browser's cache. For description, please see my humble other post
However, while using that factory and having all your session/cookies/userauth handling within those connections in sync with the WebBrowser, despite being very similar to the ClientHttp factory, you find (at least in 7.0 and 7.1 versions) that it is completely ignorant of any custom prefixes. Trying to open anything with this factory results in (WP7 v. Mango 7.1):
A first chance exception of type 'System.Net.ProtocolViolationException' occurred in System.Windows.dll
at System.Net.Browser.BrowserHttpWebRequest.InternalBeginGetRequestStream(AsyncCallback callback, Object state)
at System.Net.Browser.AsyncHelper.BeginOnUI(BeginMethod beginMethod, AsyncCallback callback, Object state)
at System.Net.Browser.BrowserHttpWebRequest.BeginGetRequestStream(AsyncCallback callback, Object state)
at MyApp.MyPage..ctor()
relevant code snippet of the MyPage:
public class WRC : IWebRequestCreate { public WebRequest Create(Uri uri) { return null;/*BREAKPOINT1*/ } }
WebRequest.RegisterPrefix("js://", new WRC()); // register the above handler
brwHttp = (IWebRequestCreate)typeof(System.Net.Browser.WebRequestCreator).GetProperty("BrowserHttp").GetValue(null, null);
var tmp = brwHttp.Create(new Uri("js://blah.blah.blah"));
var yyy = tmp.BeginGetResponse(callback, "wtf");
var response = tmp.EndGetResponse(yyy); /*BREAKPOINT2*/
var zzz = tmp.BeginGetRequestStream(callback, "wtf"); /*<---EXCEPTION*/
var stream = tmp.EndGetRequestStream(zzz); /*BREAKPOINT3*/
Execution results:
breakpoint1 never hit
breakpoint2 allows to see that "response" is NULL
breakpoint3 never hit due to the exception pasted above
My conclusion is, that the Silverlight Browser's stack is hardcoded to use some builtin set of prefixes, and all other prefixes are ignored/throw ProtocolViolation. My guess is, that in WP7 (7.0, 7.1) they are actually hardcoded to use http since my custom "js://" was passed to a BrowserHttpWebRequest.InternalBeginGetRequestStream as it's visible on the stacktrace :)
That confirms what Anthony had written - no way of having custom protocol handlers to work gracefully with the Silverlight's Browser Stack API.
However, I cannot agree with that the WebBrowser uses this connection factory. While is it true that the hidden factory is called BrowserHttp, and is true that it shares some per-user or per-session settings with the webbrowser, everything I try tens to indicate that the WebBrowser component uses yet completly other factory for its connections, and quite probably it is some native one. As an argument for that, I can only provide that I was able to successfully replace the original BrowserHttp factory with my simple custom implementation of it (both on the emulator and the phone), and with at least 6 webbrowsers in my current app, it wasn't used at all, not even once! (neither on the emulator, nor phone)

Resources