Load testing GCP Pub/Sub using JMeter

Introduction:

The Google Cloud Pub/Sub is a fully managed real-time event messaging service. It enables you to create systems of event producers and consumers, called publishers and subscribers in Google cloud platform (GCP). It helps us to build robust and scalable applications by integrating them asynchronously. It provides scalability, resilience and handles millions of messages simultaneously.

Publishers send events to the Pub/Sub topic service, without regard to how or when these events will be processed. Subscribers reads events from the Pub/Sub topic which they are subscribed to.

Compared to systems communicating through RPCs, where publishers must wait for subscribers to receive the data, such asynchronous integration increases the flexibility and robustness of the overall system. This methodology helps to Read/Write, Store and Process the real time events.

Core usages:

Pub/Sub can be used in various use cases. Some of’em were,

• Ingesting user interaction and server events
• Real-time event distribution
• Replicating data among databases
• Parallel processing and workflows
• Enterprise event bus
• Data streaming from IoT devices
• Refreshing distributed caches
• Load balancing for reliability

Key Terms:

Most of the pub/sub operations are within the below terms.

• Topic: A named resource to which messages are sent by publishers.
• Subscription: A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application. For more details about subscriptions and message delivery semantics, see the Subscriber Guide.
• Message: The combination of data and (optional) attributes that a publisher sends to a topic and is eventually delivered to subscribers.
• Message attribute: A key-value pair that a publisher can define for a message. For example, key iana.org/language_tag and value en could be added to messages to mark them as readable by an English-speaking subscriber.
• Publisher: An application that creates and sends messages to a topic(s).
• Subscriber: An application with a subscription to a topic(s) to receive messages from it.
• Acknowledgement (or “ack”): A signal sent by a subscriber to Pub/Sub after it has received a message successfully. Acked messages are removed from the subscription’s message queue.
• Push and pull: The two message delivery methods. A subscriber receives messages either by Pub/Sub pushing them to the subscriber’s chosen endpoint, or by the subscriber pulling them from the service.

Load testing Pub/Sub using Apache JMeter

Pre-requisites:

To load test the Pub/Sub service you need to have a valid credentials.json that is authorized to the project intend to test. Follow the below steps to create the credentials.json

1. Login to your GCP account
2. Goto API & Services
3. Select Credentials
4. Click on Create Credentials -> Service account
5. Enter Service account name and click create. For Service account permissions, select permissions, project, role.
6. Download the credentials.json

If you are not authorized to create credentials.json, You should reach to the respective person/team who can help you create one on your behalf.

GCP Pub-Sub plugin:

Apache JMeter provides enduser the capability to extend its feature by adding custom plugin. You need to install JMeter GCP pub/sub plugin using Plugins manager.

Setting up PubSub-PublisherConfiguration:

Once you install the GCP pubsub plugin from plugins manager. You will be able to see a config element and sampler added specifically to load test pub/sub.

You need to Add the PubSub-PublisherConfiguration to your test plan (Add-> Config -> PubSub-PublisherConfiguration). This holds the config information to connect to GCP pub/sub topic. It comprises of two section

Publisher Topic: This will help to export the connection object to the sampler. which will help us to establish a connection with multiple topics within the same test plan.

Credentials Configuration: This holds the information from the credentials.json

Setting up Publisher Request:

Sampler enables the JMeter to publish the messages to a pub/sub topic from the acquired connection using the Config element. The pub/sub publisher sampler can only fit under a ThreadGroup (Threadgroup -> Sampler-> Publisher Request)

Using Publisher request sampler, you need the below to send messages to pub/sub topic

- Publisher Topic (Mandatory) — The Pub/Sub config object name needs to be passed here. Which is created in the Pubsub Publisher configuration.

- Attributes – The Attributes which needs to be sent along with the message

- Messages(Mandatory)– This is the actual message you want to send it to the topic

Running Load Test:

Once the above configuration is successful, you can use JMeter to run a load test on pub/sub topic to pump the required events per min/hour. The testing process remains same.

Conclusion:

I personally believes that Apache JMeter has more capabilities than what it looks like. All you need is, a right — handy plugin to evaluate the performance of any application.

I have contributed the GCP Pubsub plugin to the JMeter community, which is now available in JMeter’s plugin manager. Also, If you like my work, Hit a star to the Github Repo

Will come up with another unique article. Keep supporting, Click on follow button and hit the subscribe to get the notification on new article publish.

Stay positive, Happy engineering !