I am attempting to use the grpc client for the Stackdriver/google cloud trace definition at https://github.com/googleapis/googleapis/blob/master/google/devtools/cloudtrace/v1/trace.proto
I am sending the protobuf for the
traces {
traces {
project_id: "brightcove-rna-master"
trace_id: "A096D4956A424EEB98AE7863505B1E1F"
spans {
span_id: 1
kind: RPC_CLIENT
name: "/bigtableAapiGrpcstaging_aggregated/queryDataSetInternal/buildScan"
start_time {
seconds: 1459230665
nanos: 705000000
}
end_time {
seconds: 1459230666
nanos: 416000000
}
labels {
key: "videocloud/account"
value: "4523394965001"
}
labels {
key: "videocloud/protocol"
value: "2"
}
labels {
key: "videocloud/dimensions"
value: "protocol,account"
}
}
spans {
span_id: 2
kind: RPC_SERVER
name: "/bigtableAapiGrpcstaging_aggregated/queryDataSetInternal/aggregateScan"
start_time {
seconds: 1459230666
nanos: 420000000
}
end_time {
seconds: 1459230667
nanos: 753000000
}
labels {
key: "videocloud/account"
value: "4523394965001"
}
labels {
key: "videocloud/protocol"
value: "2"
}
labels {
key: "videocloud/dimensions"
value: "protocol,account"
}
labels {
key: "bigtable/rowCount"
value: "339"
}
labels {
key: "bigtable/responseCount"
value: "136"
}
}
spans {
kind: RPC_SERVER
name: "/bigtableAapiGrpcstaging_aggregated/queryDataSetInternal"
start_time {
seconds: 1459230665
nanos: 556000000
}
end_time {
seconds: 1459230667
nanos: 754000000
}
labels {
key: "account"
value: "4523394965001"
}
}
}
}
But the only thing I get in return is this exception:
[WARN ] [2016-03-28 22:51:09,330] [grpc-default-executor-0] rna.api.server.ServerImpl Unable to send trace to google
io.grpc.StatusRuntimeException: CANCELLED
at io.grpc.Status.asRuntimeException(Status.java:431)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:358)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$3.runInContext(ClientCallImpl.java:462)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:54)
at io.grpc.internal.SerializingExecutor$TaskRunner.run(SerializingExecutor.java:154)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Similarly, I tried the following ListTraces request
traceClient.listTraces(ListTracesRequest.newBuilder()
.setProjectId(projectId)
.setView(ListTracesRequest.ViewType.COMPLETE)
.setStartTime(getEpoch())
.setEndTime(getCurrentTime())
.build());
And got:
java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNIMPLEMENTED: GRPC target method is not implemented.
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at com.brightcove.rna.api.server.ServerImpl.sendTraceAsync(ServerImpl.java:143)
at com.brightcove.rna.api.server.ServerImpl.queryDataset(ServerImpl.java:116)
at com.brightcove.rna.api.AnalyticsAPIGrpc$1.invoke(AnalyticsAPIGrpc.java:152)
at com.brightcove.rna.api.AnalyticsAPIGrpc$1.invoke(AnalyticsAPIGrpc.java:147)
at io.grpc.stub.ServerCalls$1$1.onHalfClose(ServerCalls.java:147)
at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:255)
at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$2.runInContext(ServerImpl.java:458)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:54)
at io.grpc.internal.SerializingExecutor$TaskRunner.run(SerializingExecutor.java:154)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: io.grpc.StatusRuntimeException: UNIMPLEMENTED: GRPC target method is not implemented.
at io.grpc.Status.asRuntimeException(Status.java:431)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:358)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$3.runInContext(ClientCallImpl.java:462)
... 5 more
I am pretty new to the API so I don't know what I am doing wrong here. Is there a configuration value or something similar that i missed.
Update Code that generates the spans/trace. The code represents a Grpc service that gets requests and queries a Cloud BigTable table. My intention is to trace different aspects of the request:
public class ServerImpl implements AnalyticsAPIGrpc.AnalyticsAPI {
private static final Logger logger = Logger.getLogger(ServerImpl.class);
private Connection _connection = null;
private TraceServiceFutureClient traceClient;
private String projectId;
#Override
public void queryDataset(APIRequest request, StreamObserver<APIResponse> responseObserver) {
APIResponse.Builder response = APIResponse.newBuilder();
List<TraceSpan> spans = Lists.newArrayList();
if (request.getTraceToken() != null) {
response.setTraceToken(request.getTraceToken());
}
try {
spans = queryDataSetInternal(request, response);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
} catch (Exception ex) {
responseObserver.onError(ex);
} finally {
// send trace
sendTraceAsync(request.getTraceToken(), spans);
}
}
private ListenableFuture<Empty> sendTraceAsync(String traceId, List<TraceSpan> spans) {
if (spans == null || spans.isEmpty()) {
return Futures.immediateFuture(Empty.getDefaultInstance());
}
PatchTracesRequest patchTracesRequest = PatchTracesRequest.newBuilder()
.setProjectId(projectId)
.setTraces(Traces.newBuilder().addTraces(
Trace.newBuilder()
.setProjectId(projectId)
.setTraceId(traceId.replaceAll("-", "").toUpperCase())
.addAllSpans(spans)))
.build();
if (logger.isTraceEnabled()) {
logger.trace("Sending trace: " + patchTracesRequest.toString());
}
ListenableFuture<Empty> future = traceClient.patchTraces(patchTracesRequest);
// add callback for logging result
Futures.addCallback(future, new FutureCallback<Empty>() {
#Override
public void onSuccess(#Nullable Empty result) {
logger.trace("Trace successfully sent to google");
}
#Override
public void onFailure(Throwable t) {
logger.warn("Unable to send trace to google", t);
}
});
return future;
}
private Connection getConnection() throws IOException {
return this._connection;
}
private Scan createScan(APIRequest request, String resourceName) {
return ScanBuilder.of(
request.getAccount(),
resourceName,
request.getStartTime(), request.getEndTime())
.build();
}
private List<TraceSpan> queryDataSetInternal(APIRequest request, APIResponse.Builder response) throws IOException {
AtomicLong spanIdCounter = new AtomicLong(0L);
String parentTraceName = "/api-qa/queryDataSetInternal";
TraceSpan.Builder parentSpan =
TraceSpan.newBuilder()
.setSpanId(spanIdCounter.getAndIncrement())
.setStartTime(getCurrentTime())
.setKind(TraceSpan.SpanKind.RPC_SERVER)
.setName(parentTraceName)
.putAllLabels(ImmutableMap.of("account", request.getAccount()));
Connection connection = this.getConnection();
List<TraceSpan> traceSpanList = Lists.newArrayList();
try (Table table = connection.getTable("tableName")) {
/// create scan ///
TraceSpan.Builder traceSpan = TraceSpan.newBuilder()
.setSpanId(spanIdCounter.getAndIncrement())
.setKind(TraceSpan.SpanKind.RPC_CLIENT)
.setName(parentTraceName + "/buildScan")
.setParentSpanId(parentSpan.getParentSpanId())
.setStartTime(getCurrentTime());
AtomicInteger count = new AtomicInteger(0);
// add trace span
String dimensionStr = Joiner.on(',').join(request.getDimensionsList());
traceSpan.putAllLabels(ImmutableMap.of(
"videocloud/account", request.getAccount(),
"videocloud/protocol", request.getProtocol(),
"videocloud/dimensions", dimensionStr));
// scan the response and send a stream of rows back
Scan scan = createScan(request, getResourceName(request));
logger.debug("Using bigtable scan: " + scan.toJSON());
ResultScanner scanner = table.getScanner(scan);
// record trace
traceSpanList.add(traceSpan.setEndTime(getCurrentTime()).build());
/// perform aggregation ///
Timestamp startTS = getCurrentTime();
List<Result> results = StreamSupport.stream(scanner.spliterator(), false)
.collect(Collectors.toList());
response.addAllRows(results);
// record trace
traceSpan = TraceSpan.newBuilder()
.setSpanId(spanIdCounter.getAndIncrement())
.setKind(TraceSpan.SpanKind.RPC_SERVER)
.setName(parentTraceName + "/aggregateScan")
.setParentSpanId(parentSpan.getParentSpanId())
.setStartTime(startTS)
.setEndTime(getCurrentTime())
.putAllLabels(ImmutableMap.of(
"videocloud/account", request.getAccount(),
"videocloud/protocol", request.getProtocol(),
"videocloud/dimensions", dimensionStr,
"bigtable/rowCount", String.valueOf(count.get()),
"bigtable/responseCount", String.valueOf(response.getRowsCount())));
traceSpanList.add(traceSpan.build());
response.setStatus(APIResponse.Status.OK)
.setDetails(String.format("Processed %d results from BigTable", count.get()));
} finally {
parentSpan.setEndTime(getCurrentTime());
traceSpanList.add(parentSpan.build());
}
return traceSpanList;
}
}
Ankur, I added an issue in the cloud bigtable github.
Related
I need to consume messages from a Websocket, but I have to do some logics before consume the data, so I can't use Webscoket Component.
I have a java code that do Authentication in this Websocket and subscribe a "Sensor" to receive data.
Can I create a Camel Component that I use this code in from() and every time I receive new data onNext() the Camel starts the process?
WebSocket webSocket = new WebSocket(uri, apiKey, (api, authenthication) -> {
console.println("Authenticated successfully as " + authenthication.getUserName());
String[] sensors = {sensorId};
api.getMetrics(sensors).subscribe(metrics -> {
Metric[] allMetrics = metrics.get(sensorId);
Arrays.sort(allMetrics, (metric1, metric2) -> metric1.getId().compareTo(metric2.getId()));
Metric firstMetric = allMetrics[0];
console.println("Metric: " + firstMetric.getDisplayName());
String metricId = firstMetric.getId();
String[] metric = {metricId};
api.getUnits(metric).subscribe(units -> {
Unit unit = units.get(metric[0])[0];
console.println("Unit: " + unit.getName());
Instant now = Instant.now();
Instant aMinuteAgo = now.minus(timeInterval, ChronoUnit.SECONDS);
Date start = Date.from(aMinuteAgo);
Date end = Date.from(now);
api.getData(sensorId, metricId, unit.getId(), emptyMap(), start, end).subscribe(new DisposableObserver<Data>() {
#Override
public void onNext(Data data) {
console.println("Data from last " + timeInterval + " seconds: ");
console.println(data.getData());
}
#Override
public void onComplete() {
console.println("Data update:");
Disposable subscription = api.subscribeData(sensors, metricId, unit.getId()).subscribe(updates -> {
console.println(updates.getData());
});
ScheduledExecutorService scheduler = newSingleThreadScheduledExecutor(daemonThreadFactory);
scheduler.schedule(subscription::dispose, cancelDelay, SECONDS);
}
#Override
public void onError(Throwable error) {
error.printStackTrace();
}
});
});
});
});
console.println("Connection was closed by server.");
}
Pls How can I access sqlite database on the webserver in codename one? I can only use database API to access database on the device. In order to access this on the webserver I think is quite different thing. Pls I need a snippet code on this. Thanks
Use the code below, not tested and you may have to adjust it to suite your need. Leave a comment if there's an issue:
ConnectionRequest req = new ConnectionRequest() {
#Override
protected void handleException(Exception ex) {
//handle error
}
};
req.setUrl(YourURL);
req.setPost(true);
req.setHttpMethod("POST"); //Change to GET if necessary
req.setDuplicateSupported(true);
req.addArgument("argumentToSendThroughPostOrGet1", "value1");
req.addArgument("argumentToSendThroughPostOrGet2", "value2");
NetworkManager.getInstance().addToQueueAndWait(req);
if (req.getResponseCode() == 200) {
Map<String, Object> out = new HashMap<>();
Display.getInstance().invokeAndBlock(() -> {
JSONParser p = new JSONParser();
try (InputStreamReader r = new InputStreamReader(new ByteArrayInputStream(req.getResponseData()))) {
out.putAll(p.parseJSON(r));
} catch (IOException ex) {
//handle error
}
});
if (!out.isEmpty()) {
List<Map<String, Object>> responses = (List<Map<String, Object>>) out.get("response");
for (Object response : responses) {
Map res = (Map) response;
System.out.println(res.get("key"));
}
} else {
//handle error
}
} else {
//handle error
}
TEST JSON RESPONSE:
{
"response": [
{
"key": "I was returned",
}
]
}
EDIT:
To pass data from TextField:
req.addArgument("argumentToSendThroughPostOrGet1", myTextField.getText());
Based on your comment, you can read those arguments in PHP as simple as below:
$var1 = $_POST["argumentToSendThroughPostOrGet1"];
$var1 = $_GET["argumentToSendThroughPostOrGet1"]; // if GET method is used in Codename One
//Or use $_REQUEST which supports both methods but not advisable to be used for production
...
And you can use those variables in your php code normally.
Example of Usage with MySql Query:
class Connection {
function connect() {
$mysqli = mysqli_init();
$mysqli->real_connect("localhost", "username", "password", "databaseName") or die('Could not connect to database!');
$mysqli->query("SET NAMES 'UTF8'");
return $mysqli;
}
function close() {
mysqli_close($this->connect);
}
}
$connection = new Connection();
$mysqli = $connection->connect();
$mysqli->query("SELECT * FROM MyTable WHERE ColumnName LIKE '%$var1%' ORDER BY PrimaryKeyId ASC LIMIT 100");
I am using reactive-kafka-core 0.10.1 (targeting Kafka 0.9.x). It looks like Kafka producer actor is stopped whenever an error is encountered from the callback function. Is there any way to customize this behavior? Our use case is to try to recover and resend the messages.
private def processElement(element: ProducerMessage[K, V]) = {
val record = richProducer.props.partitionizer(element.value) match {
case Some(partitionId) => new ProducerRecord(richProducer.props.topic, partitionId, element.key, element.value)
case None => new ProducerRecord(richProducer.props.topic, element.key, element.value)
}
richProducer.producer.send(record, new Callback {
override def onCompletion(metadata: RecordMetadata, exception: Exception) = {
if (exception != null) {
handleError(exception)
}
}
})
()} private def handleError(ex: Throwable) = {
log.error(ex, "Stopping Kafka subscriber due to fatal error.")
stop()
}
I'm trying to get a self-hosted Nancy app running, but I'm having trouble getting it to return valid responses. I'm new at Nancy; I expect my problem is something fairly simple.
Here's some code:
class Program
{
static void Main(string[] args)
{
const String PORT_SETTING = "webServicePortNumber";
const String URI = "http://localhost:{0}/download/";
var portNum = ConfigurationManager.AppSettings[PORT_SETTING];
var uri = new Uri(String.Format(URI, portNum));
var config = new HostConfiguration {
UrlReservations = new UrlReservations { CreateAutomatically = true }
};
using (var nancyHost = new NancyHost(new Bootstrapper(), config, uri)) {
nancyHost.Start();
Console.WriteLine(String.Format("Listening on {0}. Press any key to stop.", uri.AbsoluteUri));
Console.ReadKey();
}
Console.WriteLine("Stopped. Press any key to exit.");
Console.ReadKey();
}
}
internal class Bootstrapper : DefaultNancyBootstrapper
{
protected override Nancy.Diagnostics.DiagnosticsConfiguration DiagnosticsConfiguration
{
get {
return new DiagnosticsConfiguration {
Password = #"[password]"
};
}
}
}
My NancyModule looks like this:
public class DownloadsModule : NancyModule
{
public DownloadsModule() : base("/download")
{
RegisterRoutes();
}
private void RegisterRoutes()
{
Put["/"] = parms => InitiateDownload(parms);
Get["/"] = parms => Summary(parms);
Get["/{id}"] = parms => GetStatus(parms.requestId);
}
private Response GetStatus(Guid requestId)
{
return Response.AsText("TEST: GetStatus requestId " + requestId);
}
private Response Summary(dynamic parms)
{
return Response.AsText("Summary: You loved me before, do you love me now?");
}
private Response InitiateDownload(dynamic parms)
{
return Response.AsText("InitiateDownload.");
}
}
Nancy is running; I can access the diagnostics at http://127.0.0.1:8880/download/_Nancy/. Looking at them, the routes appear ready. Interactive Diagnostics/GetAllRoutes shows:
P U T
name: [nothing] path: /download
G E T
name: [nothing] path: /download
name: [nothing] path: /download/{id}
And yet, I'm getting 404s back when I try http://localhost:8880/download/.
The request trace on the diagnostics page shows:
Method: GET
Request Url:
Scheme: http
Host Name: localhost
Port: 8880
Base Path: /download
Path: /
Query:
Site Base: http://localhost:8880
Is Secure: false
Request Content Type:
Response Content Type: text/html
Request Headers:
<snip>
Accept: text/html;q=1
application/xhtml+xml;q=1
image/webp;q=1
application/xml;q=0.9
*/*;q=0.8
<snip>
Response Headers:
Status Code: 404
Log: New Request Started
[DefaultResponseNegotiator] Processing as real response
So why isn't Nancy routing this request to the proper route?
Problem pointed out to me by jchannon in the Nancy JabbR room:
The URI specifies http://localhost:{0}/download/, while the module also specifies a base path of /download, so currently its looking for an URL of http://localhost:{0}/download/download/
I'm trying to get the count of unread email using google API, but not able. ANy help is highly appreciated. I'm not getting any error, but the count doesnt match the actual number shown in gmail.
try
{
String serviceAccountEmail = "xxx#developer.gserviceaccount.com";
var certificate = new X509Certificate2(#"C:\Projects\xxx\xyz\API Project-xxxxx.p12", "notasecret", X509KeyStorageFlags.Exportable);
ServiceAccountCredential credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(serviceAccountEmail)
{
User = "xxx#gmail.com",
Scopes = new[] { Google.Apis.Gmail.v1.GmailService.Scope.GmailReadonly }
}.FromCertificate(certificate));
var gmailservice = new Google.Apis.Gmail.v1.GmailService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = "GoogleApi3",
});
try
{
List<Message> lst = ListMessages(gmailservice, "xxx#gmail.com", "IN:INBOX IS:UNREAD");
}
catch (Exception e)
{
Console.WriteLine("An error occurred: " + e.Message);
}
}
catch (Exception ex)
{
}
Just do: labels.get(id="INBOX") and it has those types of stats (how many messages in that label, how many are unread, and same for threads).
https://developers.google.com/gmail/api/v1/reference/users/labels/get
You can use the ListMessages method from the API example (included for completeness) for searching:
private static List<Message> ListMessages(GmailService service, String userId, String query)
{
List<Message> result = new List<Message>();
UsersResource.MessagesResource.ListRequest request = service.Users.Messages.List(userId);
request.Q = query;
do
{
try
{
ListMessagesResponse response = request.Execute();
result.AddRange(response.Messages);
request.PageToken = response.NextPageToken;
}
catch (Exception e)
{
Console.WriteLine("An error occurred: " + e.Message);
}
} while (!String.IsNullOrEmpty(request.PageToken));
return result;
}
You can use this search method to find unread messages, for example like this:
List<Message> unreadMessageIDs = ListMessages(service, "me", "is:unread");
The q parameter (query) can be all kinds of stuff (it is the same as the gmail search bar on the top of the web interface), as documented here: https://support.google.com/mail/answer/7190?hl=en.
Note that you only a few parameters of the Message objects are set. If you want to retreive the messages you'll have to use GetMessage method from the api:
public static Message GetMessage(GmailService service, String userId, String messageId)
{
try
{
return service.Users.Messages.Get(userId, messageId).Execute();
}
catch (Exception e)
{
Console.WriteLine("An error occurred: " + e.Message);
}
return null;
}
I agree that the API is not straight forward and misses a lot of functionality.
Solution for .Net:
// Get UNREAD messages
public void getUnreadEmails(GmailService service)
{
UsersResource.MessagesResource.ListRequest Req_messages = service.Users.Messages.List("me");
// Filter by labels
Req_messages.LabelIds = new List<String>() { "INBOX", "UNREAD" };
// Get message list
IList<Message> messages = Req_messages.Execute().Messages;
if ((messages != null) && (messages.Count > 0))
{
foreach (Message List_msg in messages)
{
// Get message content
UsersResource.MessagesResource.GetRequest MsgReq = service.Users.Messages.Get("me", List_msg.Id);
Message msg = MsgReq.Execute();
Console.WriteLine(msg.Snippet);
Console.WriteLine("----------------------");
}
}
Console.Read();
}