Ktor 3.0.0-rc-1 Help

First steps with Kotlin RPC

Kotlin RPC (Remote Procedure Call) is a new and exciting addition to the Kotlin ecosystem, which builds on well-established foundations and runs on the kotlinx.rpc library.

The kotlinx.rpc library enables you to make procedure calls over network boundaries using nothing but regular Kotlin language constructs. As such, it provides an alternative to both REST and Google RPC (gPRC).

In this article, we will introduce the core concepts of Kotlin RPC and build a simple application. You can then go and evaluate the library in your own projects.

Prerequisites

This tutorial assumes you have a fundamental understanding of Kotlin programming. If you are new to Kotlin, consider reviewing some introductory materials.

For the best experience, we recommend using IntelliJ IDEA Ultimate as your integrated development environment (IDE) as it offers comprehensive support and tools that will enhance your productivity.

What is RPC?

Local vs. remote procedure calls

Anyone with programming experience will be familiar with the concept of a procedure call. This is a fundamental concept in any programming language. Technically, these are local procedure calls, since they always take place within the same program.

A remote procedure call is when the function call and parameters are in some way transferred over a network, so that the implementation can occur within a separate VM/executable. The return value travels the opposite path back to the machine where the invocation was made.

It is easiest to think of the machine where the invocation occurred as the client and the machine where the implementation resides as the server. This does not necessarily have to be the case, however. RPC calls could occur in both directions, as part of a peer architecture. But to keep things simple, let’s assume a client/server deployment.

RPC framework fundamentals

Certain fundamentals must be provided by any RPC framework. They are inevitable when implementing remote procedure calls within a conventional IT infrastructure. The terminology can vary, and responsibilities can be divided in different ways, but every RPC framework must provide:

  1. A way to declare the procedures that will be invoked remotely. In object-oriented programming, an interface is the logical choice. This could be the interface construct provided by the current language or some kind of language-neutral standard, such as the Web IDL used by the W3C

  2. A means to specify the types used for parameters and return values. Once again, you could use a language-neutral standard. However, it may be simpler to annotate standard data type declarations in the current language.

  3. Helper classes, known as client stubs, that will be used to convert the procedure invocation into a format that can be sent over the network and to unpack the resulting return value. These stubs can be created either during the compiling process or dynamically at runtime.

  4. An underlying RPC Runtime that manages the helper classes and supervises the lifecycle of a remote procedure call. On the server side, this runtime will need to be embedded in some kind of server, so that it is available to process requests on an ongoing basis.

  5. Protocols need to be chosen (or defined) to represent the procedure being called, serialize the data being sent, and transform the information over the network. In the past, some technologies have defined new protocols from scratch (IIOP in CORBA), while others have focused on reuse (HTTP POST in SOAP).

Marshaling vs. serialization

In RPC frameworks, we speak of marshaling and unmarshaling. This is the process of packing and unpacking information to be sent over the network. It can be thought of as a superset of serialization. In marshaling, we are serializing objects, but we also need to package information about the procedure being called and the context in which that call was made.

Having introduced the core concepts of RPC, let’s see how they apply in kotlinx.rpc by building a sample application.

Hello, kotlinx.rpc

Let’s create an application for ordering pizza over the network. In order to keep the code as simple as possible, we’ll use a console-based client.

Create the project

First, you will create a project that will include both the client and server implementations.

In more complex applications, it's best practice to use separate modules for the client and server. However, for simplicity in this tutorial, we will use a single module for both.

  1. Launch intelliJ IDEA.

  2. On the Welcome screen, click New Project.

    Otherwise, from the main menu, select File | New | Project.

  3. In the Name field, enter KotlinRpcPizzaApp as the name of your project.

    IntelliJ New Kotlin Project window
  4. Leave the rest of the default settings and click Create.

Normally, you would immediately configure the project build file. However, that’s an implementation detail that won't enhance your understanding of the technology, so you'll get back to that step at the end.

Add the shared types

The heart of any RPC project is the interface that defines the procedures to be called remotely, along with the types used in the definition of those procedures.

In a multi-module project, these types will need to be shared. However, in this example, this step won't be necessary.

  1. Navigate to the src/main/kotlin folder and create a new subpackage called model.

  2. Inside the model package, create a new PizzaShop.kt file with the following implementation:

    package com.example.model import kotlinx.coroutines.flow.Flow import kotlinx.serialization.Serializable import kotlinx.rpc.RPC interface PizzaShop : RPC { suspend fun orderPizza(pizza: Pizza): Receipt } @Serializable class Pizza(val name: String) @Serializable class Receipt(val amount: Double)

    The interface needs to extend the RPC type from the kotlinx.rpc library. The reason for this will be explained later.

    Because you are using kotlinx.serialization to help transfer information over the network, then the types used in parameters must be marked with the Serializable annotation.

Implement the client

  1. Navigate to src/main/kotlin and create a new Client.kt file.

  2. Open Client.kt and add the following implementation:

    package com.example import com.example.model.Pizza import com.example.model.PizzaShop import io.ktor.client.* import io.ktor.http.* import kotlinx.coroutines.runBlocking import kotlinx.rpc.client.withService import kotlinx.rpc.serialization.json import kotlinx.rpc.transport.ktor.client.installRPC import kotlinx.rpc.transport.ktor.client.rpc import kotlinx.rpc.transport.ktor.client.rpcConfig fun main() = runBlocking { val ktorClient = HttpClient { installRPC { waitForServices = true } } val client: KtorRPCClient = ktorClient.rpc { url { host = "localhost" port = 8080 encodedPath = "pizza" } rpcConfig { serialization { json() } } } val pizzaShop: PizzaShop = client.withService<PizzaShop>() val receipt = pizzaShop.orderPizza(Pizza("Pepperoni")) println("Your pizza cost ${receipt.amount}") ktorClient.close() }

You only need 25 lines to prepare for and then execute an RPC call. Obviously, there is a lot going on, so let’s break the code down into sections.

The kotlinx.rpc library uses the Ktor client to host its runtime on the client side. The runtime is not coupled to Ktor, and other choices are possible, but this promotes reuse and makes it easy to integrate kotlinx.rpc into existing KMP applications.

Both the Ktor client and Kotlin RPC are built around coroutines, so you use runBlocking to create the initial coroutine, and execute the rest of the client within it:

fun main() = runBlocking { }

Next, you create an instance of the Ktor client in the standard way. kotlinx.rpc uses the WebSockets plugin under the hood to transfer information. You only need to ensure that it is loaded by using the installRPC() function:

val ktorClient = HttpClient { installRPC { waitForServices = true } }

Having created this Ktor client, you then create a KtorRPCClient object for invoking remote procedures. You will need to configure the location of the server and the mechanism being used to transfer information:

val client: KtorRPCClient = ktorClient.rpc { url { host = "localhost" port = 8080 encodedPath = "pizza" } rpcConfig { serialization { json() } } }

At this point, the standard setup has been completed, and you are ready to use the functionality specific to the problem domain. You can use the client to create a client proxy object that implements the methods of the PizzaShop interface:

val pizzaShop: PizzaShop = client.withService<PizzaShop>()

You can then make the remote procedure call and use the result:

val receipt = pizzaShop.orderPizza(Pizza("Pepperoni")) println("Your pizza cost ${receipt.amount}")

Note that a tremendous amount of work is being done for you at this point. The details of the call and all parameters must be converted into a message, sent over the network, and then the return value received and decoded. The fact that this happens transparently is the payoff for the initial setup.

Finally, we need to shut down the client as usual:

ktorClient.close()

Implement the server

The implementation on the server side breaks down into two parts. Firstly, you need to create an implementation of our interface, and secondly, you need to host it within a server.

  1. Navigate to src/main/kotlin and create a new Server.kt file.

  2. Open Server.kt and add the following interface:

    package com.example import Pizza import PizzaShop import Receipt import io.ktor.server.application.* class PizzaShopImpl( override val coroutineContext: CoroutineContext ) : PizzaShop { override suspend fun orderPizza(pizza: Pizza): Receipt { return Receipt(7.89) } }

    Obviously, this is not a real-world implementation, but it is enough to get our demo up and running. Note that we are required to accept a CoroutineContext in the constructor because our interface extends RPC, which extends CoroutineScope, which declares a coroutineContext property.

    The second part of the implementation builds on Ktor.

  3. Add the following code into the same file:

    fun main() { embeddedServer(Netty, port = 8080) { module() println("Server running") }.start(wait = true) } fun Application.module() { install(RPC) routing { rpc("/pizza") { rpcConfig { serialization { json() } } registerService<PizzaShop> { ctx -> PizzaShopImpl(ctx) } } } }

    Here's the breakdown:

    First, you create an instance of Ktor/Netty, with the specified extension function used for configuration:

    embeddedServer(Netty, port = 8080) { module() println("Server running") }.start(wait = true)

    Then, you declare a setup function that extends the Ktor Application type. This installs the kotlinx.rpc plugin and declares one or more routes:

    fun Application.module() { install(RPC) routing { } }

    Inside the routing section, you use kotlinx.rpc extensions to the Ktor Routing DSL to declare an endpoint. As with on the client, you specify the URL and configure serialization. But in this case, our implementation will be listening at that URL for incoming requests:

    rpc("/pizza") { rpcConfig { serialization { json() } } registerService<PizzaShop> { ctx -> PizzaShopImpl(ctx) } }

    Note that you use registerService to provide the implementation of your interface to the RPC runtime. You might want there to be more than a single instance, but that’s a topic for a follow-up article.

Add dependencies

You now have all the code necessary to run the application, but at the moment, it would not even compile, never mind execute. At present, there is no wizard to create a kotlinx.rpc project for us, so you need to configure the build file manually. Fortunately, this is not too complex.

  1. In the build.gradle.kts file, add the following plugins:

    plugins { kotlin("jvm") version "1.9.24" kotlin("plugin.serialization") version "1.9.24" id("io.ktor.plugin") version "2.3.12" id("com.google.devtools.ksp") version "1.9.24-1.0.20" id("org.jetbrains.kotlinx.rpc.plugin") version "0.2.1" }

    The reason for the Kotlin plugin is obvious. To explain the others:

    • The kotlinx.serialization plugin is required to generate the helper types for converting Kotlin objects into JSON. Remember that kotlinx.serialization makes no use of reflection.

    • The Ktor plugin is used to build fat JARs that bundle the app with all its dependencies.

    • KSP is the Kotlin Symbol Processing API used to develop the kotlinx.rpc plugin.

    • The RPC plugin is needed to create the stubs for the client-side.

  2. Add the following dependencies:

    dependencies { implementation("io.ktor:ktor-client-cio-jvm") implementation("org.jetbrains.kotlinx:kotlinx-rpc-krpc-client") implementation("org.jetbrains.kotlinx:kotlinx-rpc-krpc-ktor-client") implementation("io.ktor:ktor-server-netty-jvm:2.3.12") implementation("org.jetbrains.kotlinx:kotlinx-rpc-krpc-server") implementation("org.jetbrains.kotlinx:kotlinx-rpc-krpc-ktor-server") implementation("org.jetbrains.kotlinx:kotlinx-rpc-krpc-serialization-json") implementation("ch.qos.logback:logback-classic:1.5.6") testImplementation(kotlin("test")) }

    This adds the Ktor client and server, the client and server parts of the kotlinx.rpc runtime, and the library for integrating kotlinx.rpc and kotlinx-serialization.

    With this, you can now run the projects and start making RPC calls.

Run the application

To run the demo, follow the steps below:

  1. Navigate to the Server.kt file.

  2. In IntelliJ IDEA, click on the run button (intelliJ IDEA run icon) next to the main() function to start the application.

    You should see the output in the Run tool panel:

    Run server output in intelliJ IDEA
  3. Navigate to the Client.kt file and run the application. You should see the following output in the console:

    Your pizza cost 7.89 Process finished with exit code 0

Extend the example

Finally, let's enhance the complexity of our example application to establish a solid foundation for future development.

  1. In the PizzaShop.kt file, extend the orderPizza method by including the client’s ID, and add a viewOrders method that returns all the pending orders for a specified client:

    package com.example.model import kotlinx.coroutines.flow.Flow import kotlinx.serialization.Serializable import kotlinx.rpc.RPC interface PizzaShop : RPC { suspend fun orderPizza(clientID: String, pizza: Pizza): Receipt suspend fun viewOrders(clientID: String): Flow<Pizza> }

    You can take advantage of the coroutines library by returning a Flow rather than a List or Set. This will allow you to steam the information to the client one pizza at a time.

  2. Navigate to the Server.kt file and implement this functionality by storing the current orders in a map of lists:

    class PizzaShopImpl( override val coroutineContext: CoroutineContext ) : PizzaShop { private val openOrders = mutableMapOf<String, MutableList<Pizza>>() override suspend fun orderPizza(clientID: String, pizza: Pizza): Receipt { if(openOrders.containsKey(clientID)) { openOrders[clientID]?.add(pizza) } else { openOrders[clientID] = mutableListOf(pizza) } return Receipt(3.45) } override suspend fun viewOrders(clientID: String): Flow<Pizza> { val orders = openOrders[clientID] if (orders != null) { return flow { for (order in orders) { emit(order) delay(1000) } } } return flow {} } }

    Note that a new instance of PizzaShopImpl is created for each client service instance. This avoids conflicts between clients by isolating their state. However, it does not address thread safety within a single server's instance, particularly if the same instance is accessed concurrently by multiple coroutines.

  3. In the Client.kt file, submit multiple orders using two different client IDs:

    val pizzaShop: PizzaShop = client.withService<PizzaShop>() pizzaShop.orderPizza("AB12", Pizza("Pepperoni")) pizzaShop.orderPizza("AB12", Pizza("Hawaiian")) pizzaShop.orderPizza("AB12", Pizza("Calzone")) pizzaShop.orderPizza("CD34", Pizza("Margherita")) pizzaShop.orderPizza("CD34", Pizza("Sicilian")) pizzaShop.orderPizza("CD34", Pizza("California"))

    Then you iterate over the results, using the Coroutines library and streamScoped method:

    streamScoped { pizzaShop.viewOrders("AB12").collect { println("AB12 ordered ${it.name}") } pizzaShop.viewOrders("CD34").collect { println("CD34 ordered ${it.name}") } }
  4. Run the server and the client. When you run the client, you will see the results being displayed incrementally:

    Client output incrementally displaying results

Having created a working example, let’s now dig deeper into how everything works. In particular, let’s compare and contrast Kotlin RPC with the two main alternatives – REST and gRPC.

RPC vs. REST

The idea of RPC is considerably older than REST, dating back at least to 1981. When compared to REST, the RPC-based approach does not constrain you to a uniform interface (such as the HTTP request types), is much simpler to work with in code, and can be more performant thanks to binary messaging.

There are, however, three major advantages to REST:

  1. It can be used directly by JavaScript clients in the browser and, hence, as part of single-page applications. Because RPC frameworks rely on generated stubs and binary messaging, they do not fit in well with the JavaScript ecosystem.

  2. REST makes it obvious when a feature involves networking. This helps avoid the distributed objects antipattern identified by Martin Fowler. This occurs when a team splits its OO design into two or more pieces without considering the performance and reliability implications of making local procedure calls remote.

  3. REST APIs are built on a series of conventions that make them relatively easy to create, document, monitor, debug, and test. There is a huge ecosystem of tools to support this.

These trade-offs mean that Kotlin RPC is best used in two scenarios. Firstly, in KMP clients using Compose Multiplatform, and secondly, amongst collaborating microservices in the cloud. Future developments in Kotlin/Wasm may make kotlinx.rpc more applicable to browser-based applications.

Kotlin RPC vs. Google RPC

Google RPC is the dominant RPC technology in the software industry at present. A standard called Protocol Buffers (protobuf) is used to define data structures and message payloads using a language-neutral Interface Definition Language (IDL). These IDL definitions can be converted into a wide variety of programming languages and are serialized using a compact and efficient binary format. Microservice frameworks like Quarkus and Micronaut already have support for gRPC.

It would be difficult for Kotlin RPC to compete with gRPC, and there would be no benefit in this for the Kotlin community. Thankfully, there are no plans to do this. Instead, the intention is for kotlinx.rpc to be compatible and interoperable with gRPC. It will be possible for kotlinx.rpc services to use gRPC as their networking protocol and for kotlinx.rpc clients to call gRPC services. kotlinx.rpc will use its own kRPC protocol as the default option (as is the case in our current example), but there will be nothing to prevent you from choosing gRPC instead.

Next steps

Kotlin RPC extends the Kotlin ecosystem in a new direction, offering an alternative to REST and GraphQL for creating and consuming services. It is built on proven libraries and frameworks, such as Ktor, Coroutines, and kotlinx-serialization. For teams seeking to make use of Kotlin Multiplatform and Compose Multiplatform, it will provide a simple and efficient option for distributed messaging.

If this introduction has piqued your interest, make sure to check out the official kotlinx.rpc documentation and examples.

The kotlinx.rpc library is in its early stages, so we encourage you to explore it and share your feedback. Bugs and feature requests can be found on YouTrack while general discussions take place on Slack (request access).

Last modified: 09 August 2024