Can't reach GCP PubSub Emulator - google-cloud-pubsub

Question

I have setup the PubSub emulator to run a Python script locally pointing to it.
The emulator is run following the tutorial and it seems to be working fine:
➜ gcloud beta emulators pubsub start --project=tokyo-rain-123 --host-port=127.0.0.1:8085
Executing: /Users/pabloruiz/Downloads/google-cloud-sdk/platform/pubsub-emulator/bin/cloud-pubsub-emulator --host=127.0.0.1 --port=8085
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub] Nov 17, 2022 3:32:04 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[pubsub] SLF4J: Defaulting to no-operation (NOP) logger implementation
[pubsub] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[pubsub] Nov 17, 2022 3:32:04 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: Server started, listening on 8085
It is working fine using curl:
➜ curl -X PUT http://localhost:8085/v1/projects/tokyo-rain-123/topics/topic-data1
{
"name": "projects/tokyo-rain-123/topics/topic-data1"
}
Although it is displaying this warning on the emulator:
[pubsub] INFO: Server started, listening on 8085
[pubsub] Nov 17, 2022 4:00:57 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected non-HTTP/2 connection.
[pubsub] Nov 17, 2022 4:00:57 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected HTTP/2 connection.
I can't reach it using Postman. I receive this response:
{
"error": {
"code": 500,
"message": "RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR",
"status": "INTERNAL"
}
}
And this is the log from the emulator:
[pubsub] WARNING: Metadata key is 'Connection', which should not be used. That is used by HTTP/1 for connection-specific headers which are not to be forwarded. There is probably an HTTP/1 conversion bug. Simply removing the Connection header is not enough; you should remove all headers it references as well. See RFC 7230 section 6.1
[pubsub] java.lang.RuntimeException: exception to show backtrace
[pubsub] at io.grpc.Metadata$Key.validateName(Metadata.java:740)
[pubsub] at io.grpc.Metadata$Key.<init>(Metadata.java:762)
[pubsub] at io.grpc.Metadata$Key.<init>(Metadata.java:671)
[pubsub] at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:971)
[pubsub] at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:966)
[pubsub] at io.grpc.Metadata$Key.of(Metadata.java:708)
[pubsub] at io.grpc.Metadata$Key.of(Metadata.java:704)
[pubsub] at io.gapi.emulators.grpc.HttpAdapter$StubMethodHandler.handle(HttpAdapter.java:537)
[pubsub] at io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler.handle(HttpAdapter.java:572)
[pubsub] at io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:195)
[pubsub] at io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
[pubsub] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
[pubsub] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
[pubsub] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
[pubsub] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[pubsub] at java.base/java.lang.Thread.run(Thread.java:1589)
[pubsub]
[pubsub] Nov 17, 2022 4:02:20 PM io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener shouldIgnoreHeadersOrDataFrame
[pubsub] INFO: [id: 0xde374c44, L:/127.0.0.1:56005 - R:/127.0.0.1:56628] ignoring DATA frame for stream RST_STREAM sent.
[pubsub] Nov 17, 2022 4:02:20 PM io.gapi.emulators.netty.HttpHandler$1 onError
[pubsub] INFO: Exception when handling request: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR
If I try it with the Python client, it just never reaches the server, since I can't see anything in the logs. This is the Python scripts:
import logging
import os
from dotenv import load_dotenv
from google.cloud import pubsub_v1
load_dotenv()
logger = logging.getLogger(__name__)
HOST, PROJECT_ID = os.environ.get("PUBSUB_EMULATOR_HOST"), os.environ.get("PUBSUB_PROJECT_ID")
print(f"Host: {HOST}, Project ID: {PROJECT_ID}")
publisher = pubsub_v1.PublisherClient()
def list_topics(project_id):
project_path = f"projects/{project_id}"
all_topics = list(publisher.list_topics(
request={"project": project_path},
timeout=5
))
print(f"Found {len(all_topics)}")
return all_topics
logger.info("Listing topics for project:")
topics = list_topics(project_id=PROJECT_ID)
for topic in topics:
logger.info(topics)
which is just printing the correct env variables, but the client never gets a response for the rpc call.
Host: [::1]:8085, Project ID: tokyo-rain-123
Any idea why this is happening ?
Thanks in advance !

Answer 1

Can't comment, but found a relevant issue tracked by google: https://issuetracker.google.com/issues/259505258

Answer 2

It seems like you are starting the emulator on --host-port=127.0.0.1:8085, not [::1]:8085. If you specify the Host as localhost:8085, it should work.

Answer 3

Through Postman, uncheck the Connection key.
The reason why it works through curl is because you don't have Connection metadata.