Stackdriver log grouping ignoring in the monitored resource definition - google-app-engine

We've set up a "parent-child" relationship for the logging of our system as per this documentation. We're also extracting the traceId for the X-Cloud-Trace-Context header using the format defined here.
The logs are grouped nicely in the Stackdriver console when dealing with only one service, but a trace can span multiple services. The behaviour we're seeing is the child logs from all services are bundled with the request log, even if we filter the view via the console. I would have thought that the resource definition should have helped filter this view in the console? i.e. If we filtered to only see service B, we shouldn't see child logs associated with service A?
Here's a snippet of code that constructs the parent and child
import (
"cloud.google.com/go/logging"
"google.golang.org/genproto/googleapis/api/monitoredres"
"os"
"time"
)
//ParentLogger constructs a logger to bundle child log lines to
func ParentLogger(client *logging.Client) *logging.Logger {
initParentLoggerOnce.Do(func() {
commonResource := &monitoredres.MonitoredResource{
Labels: map[string]string{
"module_id": os.Getenv("GAE_SERVICE"),
"project_id": os.Getenv("GOOGLE_CLOUD_PROJECT"),
"version_id": os.Getenv("GAE_VERSION"),
},
Type: "gae_app",
}
parentLogger = client.Logger(
"log_parent",
logging.CommonLabels(map[string]string{}),
logging.CommonResource(commonResource),
)
})
return parentLogger
}
//ChildLogger constructs a logger representing individual log lines
func ChildLogger(client *logging.Client) *logging.Logger {
initChildLoggerOnce.Do(func() {
commonResource := &monitoredres.MonitoredResource{
Labels: map[string]string{
"module_id": os.Getenv("GAE_SERVICE"),
"project_id": os.Getenv("GOOGLE_CLOUD_PROJECT"),
"version_id": os.Getenv("GAE_VERSION"),
},
Type: "gae_app",
}
childLogger = client.Logger(
"log_child",
logging.CommonLabels(map[string]string{}),
logging.CommonResource(commonResource),
)
})
return childLogger
}
//LogRequest posts the parent request log to stackdriver
func (l *gcpRequestLogger) LogRequest() error {
var rSize int64 = 0 //simplified for stack overflow post
rStatus := 200 //simplified for stack overflow post
//
traceID, spanID, sampled := deconstructTraceContext(l.r.Header.Get("X-Cloud-Trace-Context"))
l.parentLogger.Log(logging.Entry{
HTTPRequest: &logging.HTTPRequest{
Request: l.r,
Latency: time.Since(l.start),
ResponseSize: rSize,
Status: rStatus,
RemoteIP: l.r.RemoteAddr,
},
Timestamp: time.Now().UTC(),
Severity: l.severity,
Trace: traceID,
SpanID: spanID,
Payload: l.requestLogPayload,
TraceSampled: sampled,
})
return nil
}
While we could start a new trace per request on a microservice for useful request log grouping but we'd then lose the nice global view in Stackdriver Trace (not to mention the logs correlated to the trace). Have we done something wrong, or is this a gap in the way Stackdriver Logging working?
UPDATE:
Added image of behaviour we're seeing. The last group of logs were not expected as they were generated our config service and in this view I'm filtering by the support-portal service.

Related

Getting message: "Match failed [400]", errorType: "Meteor.Error" when using rocketchat methods(loadHistory) in Reactjs

I am trying to integrate Rocket chat to my application using asteroid but when I use Rocket.chat's LoadHistory method I get {isClientSafe: true, error: 400, reason: "Match failed", message: "Match failed [400]", errorType: "Meteor.Error"}. To be clear, I have never used Rocket.chat or Asteroid before. However some methods are working flawlessly.
/**
* Calls a server-side method with the specified arguments.
* #param method string required: the name of the method to call
* #param params [param1, param2, ...] any optional: parameters passed to the
server method
* #return Promise resolved, rejected
*/
let asteroidMethods = function (method, param){
return socket.call(method, param)
};
getMessageHistory = (lastMessage) => {
let param =[ lastMessage.rid, null, 50, lastMessage['ts']['$date']];
asteroidMethods("loadHistory", param).then(res=>{
console.log(res)
}).catch(e=>{
console.log(e)
})
}
Since the Match failed response is a rather generic one, there're several potential issues causing your error. For example, the lastMessage.rid could be empty.
With your problem, the Rocket.Chat server logs shall give you more information about the underlaying problem. For that, Rocket.Chat offers a nice way of accessing the server logs through the user interface:
Log in to the Rocket.Chat admin UI as admin user
Ensure the Log Level to be very verbose -> Level 2 (https://example.com/admin/Logs)
Visit the log view (https://example.com/admin/view-logs)
Execute your code once
Example
I've set my roomId to 42 in an example request:
connection.method('loadHistory', [42, roomId, oldestMessageDate, 15])
That triggers the following log event with very detailed information:
Exception while invoking method 'loadHistory'
{ Error: Match error: Expected string, got number }
With that information, you should be able to identify the actual issue behind your request.

Request Deferrer with Service Worker in PWA

I am making a PWA where users can answer the forms. I want it to make also offline, so when a user fills out a form and does not have the internet connection, the reply will be uploaded when he is back online. For this, I want to catch the requests and send them when online. I wanted to base it on the following tutorial:
https://serviceworke.rs/request-deferrer_service-worker_doc.html
I have managed to implement the localStorage and ServiceWorker, but it seems the post messages are not caught correctly.
Here is the core function:
function tryOrFallback(fakeResponse) {
// Return a handler that...
return function(req, res) {
// If offline, enqueue and answer with the fake response.
if (!navigator.onLine) {
console.log('No network availability, enqueuing');
return;
// return enqueue(req).then(function() {
// // As the fake response will be reused but Response objects
// // are one use only, we need to clone it each time we use it.
// return fakeResponse.clone();
// });
}
console.log("LET'S FLUSH");
// If online, flush the queue and answer from network.
console.log('Network available! Flushing queue.');
return flushQueue().then(function() {
return fetch(req);
});
};
}
I use it with:
worker.post("mypath/add", tryOrFallback(new Response(null, {
status: 212,
body: JSON.stringify({
message: "HELLO"
}),
})));
The path is correct. It detects when the actual post event happens. However, I can't access the actual request (the one displayed in try or fallback "req" is basically empty) and the response, when displayed, has the custom status, but does not contain the message (the body is empty). So somehow I can detect when the POST is happening, but I can't get the actual message.
How to fix it?
Thank you in advance,
Grzegorz
Regarding your sample code, the way you're constructing your new Response is incorrect; you're supplying null for the response body. If you change it to the following, you're more likely to see what you're expecting:
new Response(JSON.stringify({message: "HELLO"}), {
status: 212,
});
But, for the use case you describe, I think the best solution would be to use the Background Sync API inside of your service worker. It will automatically take care of retrying your failed POST periodically.
Background Sync is currently only available in Chrome, so if you're concerned about that, or if you would prefer not to write all the code for it by hand, you could use the background sync library provided as part of the Workbox project. It will automatically fall back to explicit retries whenever the real Background Sync API isn't available.

Google App Engine Datastore, returning before it was updated for several seconds

So I have values I need to update in my datastore. I am using a transaction do so as seen below. After the update has been committed I send a result back to the client letting them know the update has went through. The client then sends another request for an updated list of items. All code executes correctly as far as I can tell, no errors thrown and eventually I do get the update requested showing as expected.
My issue is even after the commit sometimes it is several seconds sometimes before the update shows in the returned list. If it was just a delay that would be crappy, but it is worse than that. The list returns incorrect / non-updated values during this time period. I understand that there can be delays with this architecture, but I thought the entire point of transactions was so that if something was updated like this nothing could possibly read the old value once the transaction grabbed the desired item? To see the old value for a long, long time afterwards seems incorrect. Much less waiting until after the transaction says it has been committed and having an entire 300ms+ RTT and still getting bad values for several seconds after it was supposedly committed. What am I missing here?
/*
We don't actually delete a post, we simply replace it with a [removed]
version of itself.
*/
router.delete('/', function (req, res) {
//Check our parameters
if (req.body == null || req.body["Thread"] == null || typeof req.body["Thread"] !== "string") {
res.json({ success: false, message: "Thread name was not included as a parameter.", data: null});
return;
}
if (req.body == null || req.body["PostNumber"] == null) {
res.json({ success: false, message: "Post Number was not included as a parameter.", data: null });
return;
}
if ((parseInt(req.body["PostNumber"]) || 0) < 1) {
res.json({ success: false, message: "Post Number was not a valid numeric value.", data: null });
return;
}
var transaction = datastore.transaction();
transaction.get(datastore.key(['Post', PostName(req.body["Thread"], 6, parseInt(req.body["PostNumber"]))]), function (err, value) {
if (err)
{
res.json({ success: false, message: "Transaction failed.", data: null });
return;
}
if (value === null)
{
res.json({ success: false, message: "Post and thread combination does not exist.", data: null });
return;
}
value.data.CreatorName = "[removed]";
value.data.CreatorFooter = "";
value.data.Contents = "[removed]";
transaction.save(value);
transaction.commit(function (err) {
if (err)
{
res.json({ success: false, message: "Transaction failed.", data: null });
}
else
{
res.json({ success: true, message: "Erased post information from table", data: null });
}
});
});
});
What you experience is called "eventual consistency", and it is an important part of Cloud Datastore architecture. Without it the Datastore would be much slower for all requests.
Note that all get operations are always consistent - only queries are affected as it takes time to update all indexes. Updates to indexes may take up to a few seconds.
There are several strategies to work with eventual consistency, but they are not really applicable/necessary in your use case, because the updated data is already available to your client app. It was your client app that initiated the save request. In almost all situations it means that you can avoid reloading the data.
For example, if an app displays a list of 17 records and a user added a new record, you can simply add the new record object to the displayed list after the backend responds to the save request. Such response may include data that is missing from the record being saved, e.g. its datastore ID. In some situations it may be easier to return the entire saved object if it has many properties that have been updated on the server side.
In a very rare case when saving an object requires the client to load a totally new set of data, which may include the updated object, the client may still replace the object returned in a query with an updated object - or add it if it is missing. Again, by the time the query results arrive you already have a complete updated object from your "save object" response.

403 Unauthorized when updating Meteor User

I'm getting a 403 when trying to update a user's profile using meteor-angular. Unfortunately, it's not very descriptive -- the complete error is:
{
details: undefined
error: 403
errorType: "Meteor.Error"
message: "Access denied [403]"
reason: "Access denied"
}
I'm under the impression that I don't need to add anything to the server side, but I added to try to get some visibility into what the actual update request looked like.
Meteor.users.deny {
update: (userId, user, fields, modifier) ->
console.log("meteor deny!")
console.log(userId, user._id)
console.log(fields, modifier)
false
}
Meteor.users.allow {
update: (userId, user, fields, modifier) ->
console.log("allow", arguments)
true
}
For debugging. In the logs, I see
I20150707-22:14:22.955(-6)? meteor deny!
I20150707-22:14:22.956(-6)? Hk83p9hieEBYHhzo6 Hk83p9hieEBYHhzo6
I20150707-22:14:22.956(-6)? [ 'profile' ] { '$set': { 'profile.name': 'ben', 'profile.gender': 'male' } }
Which is exactly what I would expect to see and appears to be what is required when looking at the docs -- i.e. a user is editing their own profile and the $set is using dot notation. I'm triggering the update from the client, basically using pretty plain angular. The interesting bits are
....
$scope.user = $meteor.object $meteor.getCollectionByName("users"), $stateParams.userId
$scope.save = () ->
if $scope.form.$valid
$scope.user.save({ 'profile.name': $scope.user.profile.name, 'profile.gender': $scope.user.profile.gender}).then(
(numberOfDocs) ->
console.log 'save successful, docs affected ', numberOfDocs
(error) ->
console.log 'save error ', error
)
I'm sort of at a loss for what to try next. My packages are
meteor-platform
urigo:angular
angularui:angular-ui-router
coffeescript
mquandalle:stylus
civilframe:angular-jade
twbs:bootstrap
angularui:angular-ui-bootstrap
mquandalle:bower
tmeasday:publish-counts
aldeed:collection2
angularutils:pagination
accounts-ui
okland:accounts-phone
accounts-base
Not a full answer, but to fix your problem immediately, just create a method that updates a users profile server side. Just check that a user is only trying to edit his own profile.

Backbone.js save error

I am scratching my head over an issue with backbone...
I have a backbone view, with an event which calls a function that saves the current model.
Its a simple form, in the view I do some validation before syncing the model.
this.model.save({
completed : completed,
company : company,
revenue : revenue,
term : term,
comments : comments,
probability : probability
},
{
success: function (model, response) {
console.log('success', model, response);
Evision.trackRouter.navigate("tracker/", {trigger: true});
},
error: function (model, response) {
console.log('error', model, response);
}
}
;
After success is fired it returns me back to my collection of models, where I can select another model to edit, its at this point when i attempt to save that i receive in my console and Chrome crashes.
If i expand this error it is indicating a problem with the function running the above, and my model is logged out in the console but when i try to expand it its empty?
Its worth saying that neither success or error is being fired ont he 2nd route round.
Thanks
Edit #1
Here is the error i receive before the crash
<error>
w.extend
w.clone
e.extend.toJSON
_.extend.save
_.extend.update
LocalStorage.sync
Backbone.sync
e.extend.sync
e.extend.save
Evision.Views.TrackerDetail.Backbone.View.extend.saveTracker
(anonymous function)
j
Edit #2 Here is my model
Evision.Models.Track = Backbone.Model.extend({
defaults: function() {
return {
id : Evision.trackerList.nextOrder(),
completed : false,
created : Utils.datestamp(),
company : "",
revenue : "",
term : "",
comments : "",
probability : "",
success : null
}
}
});
I know exactly the problem. I had this issue not so long back. First thing I looked at was the stack trace and just like in the comments above I was getting an extend problem with underscore. What you need to do is update backbone.js, underscore.js and backbone-localstorage.js(if you have it) to the most recent versions. The problem lies with versioning!
Turns out the issue was related to a modified version of Jerome's Backbone.localStorage, which would allow for both remote and local storage within the app. I replaced with the latest localStorage adapter and everything is working fine. Unfortunately I cant find the original source to notify.

Resources