Changelog 2.2 version
2.2.4
released 28th February 2023
Client
ContentNegotiation: The "charset=UTF-8" part is added for the Content-Type header
To reproduce run the following code:
val client = HttpClient(Apache) {
install(ContentNegotiation) {
json()
}
}
val response = client.post("https://httpbin.org/post") {
contentType(ContentType.Application.Json)
setBody(123)
}.bodyAsText()
println(response)
I expect a request Content-Type
header to be application/json
(like in 1.6.7) but actually it's application/json; charset=UTF-8
.
The changed behavior unnecessary breaks users' applications after migration to 2.0.0 (https://kotlinlang.slack.com/archives/C0A974TJ9/p1644027042441219).
Connect timeout is not respected when using the HttpRequestRetry plugin
Issue
If the HttpTimeout
plugin is used in parallel with the HttpRequestRetry
plugin the connectTimeoutMillis
configuration is not respected and the request enters the retry loop.
If the HttpTimeout
plugin is installed after HttpRequestRetry
the requestTimeoutMillis
configuration is not respected as well.
Reproducing the problem
Here you can find a basic example that reproduces the problem.
Platform
Noticed on JVM
OkHttp: Cancelling while writing to ByteWriteChannel when overriding WriteChannelContent causes propagation of CancellationException to a caller
Hello Ktor team :)
It has been pleasure to work with Ktor so far, but we stumbled on some issue that we are not sure how to solve, I will try to provide as much information as I can but if you need anything else from me please let me know so that perhaps we can figure something out together :)
We are trying to implement the feature where we want to cancel an outgoing asset upload to the server, this is basicly what we are trying to achieve :
class StreamAssetContent(
private val fileContentStream: okio.Source
) : OutgoingContent.WriteChannelContent() {
override suspend fun writeTo(channel: ByteWriteChannel) {
val contentBuffer = Buffer()
while (fileContentStream.read(contentBuffer, BUFFER_SIZE) != -1L) {
contentBuffer.readByteArray().let { content ->
channel.writePacket(ByteReadPacket(content))
}
}
channel.writeStringUtf8(closingArray)
channel.flush()
channel.close()
}
private companion object {
const val BUFFER_SIZE = 1024 * 8L
}
}
suspend fun cancelDuringUpload() {
val httpClient = HttpClient(engine = OkHttp.create())
val uploadJob = launch {
httpClient.post(PATH_PUBLIC_ASSETS_V3) {
contentType(ContentType.MultiPart.Mixed)
setBody(StreamAssetContent(fileSource))
}
}
delay(100.milliseconds)
uploadJob.cancel()
}
according to my knowledge this should work as it cancels the coroutine it is launched in, but it crashes the app with CancellationException, we were able to solve
by isolating the throwing of the CancellationException from the insides of the Ktor
class StreamAssetContent(
private val fileContentStream: okio.Source
) : OutgoingContent.WriteChannelContent() {
override suspend fun writeTo(channel: ByteWriteChannel) {
try {
coroutineScope {
if (!channel.isClosedForWrite && producerJob.isActive) {
channel.writeStringUtf8(openingData)
val contentBuffer = Buffer()
while (fileContentStream.read(contentBuffer, BUFFER_SIZE) != -1L) {
contentBuffer.readByteArray().let { content ->
channel.writePacket(ByteReadPacket(content))
}
}
channel.writeStringUtf8(closingArray)
channel.flush()
channel.close()
}
}
} catch (e: Exception) {
channel.flush()
channel.close()
producerJob.completeExceptionally(e)
throw IOException(e.message)
} finally {
producerJob.complete()
}
}
}
This would be a nice fix, but unfortunetly this was not enough, because when the timing is right the content of this class coming from Ktor also can throw CancellationException
internal class StreamRequestBody(
private val contentLength: Long?,
private val block: () -> ByteReadChannel`
) : RequestBody() {
override fun contentType(): MediaType? = null
override fun writeTo(sink: BufferedSink) {
block().toInputStream().source().use {
sink.writeAll(it)
}
}
override fun contentLength(): Long = contentLength ?: -1
override fun isOneShot(): Boolean = true
}
so we again override it and this is we come up with :
class ByteChannelRequestBody(
private val contentLength: Long?,
private val callContext: CoroutineContext,
private val block: () -> ByteReadChannel
) : RequestBody(), CoroutineScope {
private val producerJob = Job(callContext[Job])
override val coroutineContext: CoroutineContext
get() = callContext + producerJob + Dispatchers.IO
override fun contentLength(): Long = contentLength ?: -1
override fun contentType(): MediaType? = null
override fun writeTo(sink: BufferedSink) {
withJob(producerJob) {
if (producerJob.isActive) {
block().toInputStream().source().use {
sink.writeAll(it)
}
}
}
}
private inline fun <T> withJob(job: CompletableJob, block: () -> T): T {
try {
return block()
} catch (ex: Exception) {
job.completeExceptionally(ex)
// wrap all exceptions thrown from inside `okhttp3.RequestBody#writeTo(..)` as an IOException`
throw IOException(ex)
} finally {
job.complete()
}
}
}
the issue here is that in order to use this imlementation I had to copy-paste Ktor source code, because the place that is allowing us to use our custom Class is Ktor internal. This requires us to maintain the Ktor classes with each update, which we would ike not to do since you guys are doing it way better :P
internal fun OutgoingContent.convertToOkHttpBody(callContext: CoroutineContext): RequestBody = when (this) {
is OutgoingContent.ByteArrayContent -> bytes().let {
it.toRequestBody(null, 0, it.size)
}
is OutgoingContent.ReadChannelContent -> ByteChannelRequestBody(contentLength, callContext) { readFrom() }
is OutgoingContent.WriteChannelContent -> {
ByteChannelRequestBody(contentLength, callContext) {
GlobalScope.writer(callContext) { writeTo(channel) }.channel
}
}
is OutgoingContent.NoContent -> ByteArray(0).toRequestBody(null, 0, 0)
else -> throw UnsupportedContentTypeException(this)
}
We are on :
okio = 3.2.0
ok-http = 4.9.3
ktor = 2.2.1
Here is also the stacktrace when the issues are happening:
com.wire.kalium.network.exceptions.KaliumException$GenericError
at com.wire.kalium.network.api.v2.authenticated.AssetApiV2.uploadAsset$suspendImpl(AssetApiV2.kt:89)
at com.wire.kalium.network.api.v2.authenticated.AssetApiV2$uploadAsset$1.invokeSuspend(Unknown Source:18)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.util.concurrent.CancellationException: test
at com.wire.kalium.network.http.request.ByteChannelRequestBody.writeTo(ByteChannelRequestBody.kt:63)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:59)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
at java.lang.Thread.run(Thread.java:1012)
so my question to you, is there anything we are perhaps doing wrong, or is this simply not yet easily supported in Ktor ?
There is also an issue on this from AWS :
https://github.com/awslabs/aws-sdk-kotlin/issues/733
And this is the PR coming from us - Wire
https://github.com/wireapp/kalium/pull/1288
thanks for any help and let us know if you need anything else from us
Core
URLs with underscore fail to parse correctly in HTTP client request
Unable to use underscores in request URL with Ktor http client. Client fails to parse port and name correctly urls with underscores.
For example http://my_service:8080
fails to parse correctly. Calling URL("http://my_service:8080")
on seems to parse port and hostname but calling toURI()
on it for seems to fail to parse port and hostname correcly. This is an issue with URLBuilder.takeFrom(url: URL): URLBuilder = takeFrom(url.toURI())
This seems to be somewhat known issue with Java URI class.
Hostnames with underscores are quite common in docker environments and crucial for my use case. Is this something you would be looking to support?
Link to related method: https://github.com/ktorio/ktor/blob/0081f943b434bdd0afd82424b389629e89a89461/ktor-http/jvm/src/io/ktor/http/URLUtilsJvm.kt#L49
Generator
Create MongoDB Plugin for Generator
The Plugin should be listed in the Generator as a MongoDB ${used library}
plugin with some short description and code snippet. On apply it should add:
- required dependencies
- simple database model with initialization
- 4 CRUD endpoints
Update the latest version of Ktor by the latest version of Ktor Gradle plugin
We have a problem after releasing a new version of Ktor; there is a time gap between the release of Ktor and the Ktor Gradle plugin, so a new project with the latest released version cannot be built because of the absence of the respective version of Ktor Gradle plugin.
Create Postgres JDBC Plugin for Generator
The Plugin should be listed in the Generator as a Postgres
plugin with some short description and code snippet. On apply it should add:
- required dependencies
- simple database model with initialization
- 4 CRUD endpoints
Create Exposed Plugin for Generator
The Plugin should be listed in the Generator as an Exposed
plugin with some short description and code snippet. On apply it should add:
- expose dependencies
- simple database model with initialization
- 4 CRUD endpoints
IntelliJ IDEA Plugin
OpenApi for StatusPages is cached and not updated accordingly to the code
- Type
fun Application.configureRouting() {
install(StatusPages) {
exception<IllegalStateException> { call, cause ->
call.respond(HttpStatusCode.BadGateway, "This is response")
}
}
routing {
get("/") {
call.respondText("Hello World!")
}
get("/{name}") {
throw IllegalStateException("FOO")
}
}
}
- Generate OpenAPI
- Remove
exception
handler:
fun Application.configureRouting() {
install(StatusPages) {
}
routing {
get("/") {
call.respondText("Hello World!")
}
get("/{name}") {
throw IllegalStateException("FOO")
}
}
}
- Generate OpenAPI.
Expected: Now OpenAPI for /{name}
does not reflect response HttpStatusCode.BadGateway, "This is response"
now.
Actual: It hasn't changed.
NullPointerException in KtorRoutingVisitor.processLastArgumentsBodyIfRouteExtensionLambda
at io.ktor.ide.KtorRoutingVisitor.processLastArgumentsBodyIfRouteExtensionLambda(KtorUrlResolver.kt:149)
at io.ktor.ide.KtorRoutingVisitor.visitCallExpression(KtorUrlResolver.kt:188)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:165)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:169)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:169)
at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings$lambda$3$lambda$2(KtorUrlResolver.kt:65)
at io.ktor.ide.KtorUtilsKt.withProgress(KtorUtils.kt:112)
at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings$lambda$3(KtorUrlResolver.kt:64)
at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:39)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$3(CachedValueBase.java:231)
at com.intellij.util.CachedValueBase.computeData(CachedValueBase.java:41)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$4(CachedValueBase.java:231)
at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:112)
at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:66)
at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:232)
at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:28)
at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:72)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:111)
at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings(KtorUrlResolver.kt:61)
at io.ktor.ide.KtorUrlResolverKt.access$getAllUrlMappings(KtorUrlResolver.kt:1)
at io.ktor.ide.KtorUrlResolver$getVariants$2.invoke(KtorUrlResolver.kt:41)
at io.ktor.ide.KtorUrlResolver$getVariants$2.invoke(KtorUrlResolver.kt:41)
at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:315)
at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)
at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
at kotlin.sequences.FilteringSequence$iterator$1.calcNext(Sequences.kt:169)
at kotlin.sequences.FilteringSequence$iterator$1.hasNext(Sequences.kt:194)
at com.intellij.microservices.gotosymbol.UrlSearchEverywhereContributor$fetchWeightedElements$lambda$2$$inlined$computeInReadActionWithWriteActionPriority$1.run(UrlSearchEverywhereContributor.kt:205)
at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1086)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:71)
at com.intellij.openapi.progress.util.ProgressIndicatorUtilService.runActionAndCancelBeforeWrite(ProgressIndicatorUtilService.java:63)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:128)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:109)
at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:68)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:589)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:664)
at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:620)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:588)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:68)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:106)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:71)
at com.intellij.microservices.gotosymbol.UrlSearchEverywhereContributor.p(UrlSearchEverywhereContributor.kt:202)
at com.intellij.concurrency.JobLauncherImpl.lambda$processImmediatelyIfTooFew$2(JobLauncherImpl.java:137)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:589)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:664)
at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:620)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:588)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
at com.intellij.concurrency.JobLauncherImpl.lambda$processImmediatelyIfTooFew$3(JobLauncherImpl.java:133)
at com.intellij.concurrency.JobLauncherImpl.processImmediatelyIfTooFew(JobLauncherImpl.java:147)
at com.intellij.concurrency.JobLauncherImpl.invokeConcurrentlyUnderProgress(JobLauncherImpl.java:44)
OpenAPI for StatusPages: java.lang.IllegalStateException: Calling invokeAndWait from read-action leads to possible deadlock
232.1708
fun Application.configureRouting() {
install(StatusPages) {
exception<IllegalStateException> { call, cause ->
call.respond(HttpStatusCode.BadGateway, "500")
}
}
routing {
get("/") {
call.respondText("Hello World!")
}
get("/{name}") {
throw IllegalStateException("FOO")
// call.respond("OK")
// val name = call.parameters["name"] ?:
// call.respondText("Hello $name!")
}
}
}
Exception:
java.lang.IllegalStateException: Calling invokeAndWait from read-action leads to possible deadlock.
at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:454)
at com.intellij.openapi.application.ex.ApplicationUtil.invokeAndWaitSomewhere(ApplicationUtil.java:145)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcessWithProgressSynchronously(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.runProcessWithProgressSynchronously(ProgressManagerImpl.java:85)
at com.intellij.openapi.progress.impl.CoreProgressManager.run(CoreProgressManager.java:367)
at com.intellij.openapi.progress.ProgressManager.run(ProgressManager.java:208)
at io.ktor.ide.actions.test.TestActionsUtils$Companion.executeWithModalProgress(TestActionsUtils.kt:65)
at io.ktor.ide.actions.test.TestActionsUtils$Companion.findParentRoutingCall$intellij_ktor_starter(TestActionsUtils.kt:213)
at io.ktor.ide.actions.test.TestActionsUtils$Companion.findKtorApplicationConfig(TestActionsUtils.kt:116)
at io.ktor.ide.oas.KtorStatusPagesOasSupportKt.findStatusPagesMapping(KtorStatusPagesOasSupport.kt:22)
at io.ktor.ide.oas.KtorOasVisitor.findStatusPagesMappingCached$lambda$6(KtorOasVisitor.kt:120)
Ktor IDE does not generate proper OpenAPI with StatusPages and recursive calls (plus `error(...)` calls)
Ex:
install(StatusPages) {
exception<Throwable> { call, cause ->
call.respondText(text = "500: ${'$'}cause", status = HttpStatusCode.InternalServerError)
}
exception<IllegalStateException> { call, cause ->
call.respond("AAA")
}
}
...
get("/eee") {
error("aaa")
myError()
}
...
fun myError(): Nothing = throw Exception()
In this case, /eee
route in OpenAPI does not represent handlers from StatusPages because myError
is recursive call that throws (although it is not Route. or Call. extension)
Samples
Integration tests for API secured with (RSA signed) JSON web tokens sample
Apply Ktor Gradle plugin to all sample projects and remove specific versions for artifacts
If the Ktor Gradle plugin is applied to the project, the specific versions for Ktor's dependencies are redundant.
Server
Routing: Wrong content-type results in 405 instead of 415 status code with two routes
In continuation of KTOR-4849. The example provided there is indeed fixed in 2.2.3. However, if we add another route, another issue emerges. Instead of 415 I get 405.
package com.example
import io.ktor.http.ContentType
import io.ktor.http.HttpMethod
import io.ktor.server.application.Application
import io.ktor.server.application.call
import io.ktor.server.application.log
import io.ktor.server.engine.embeddedServer
import io.ktor.server.netty.Netty
import io.ktor.server.response.respond
import io.ktor.server.routing.accept
import io.ktor.server.routing.contentType
import io.ktor.server.routing.route
import io.ktor.server.routing.routing
fun main() {
embeddedServer(Netty, port = 3000) {
module()
}.apply {
start(wait = true)
}
}
fun Application.module(): Unit {
routing {
route("/some/plain/path", HttpMethod.Post) {
contentType(ContentType.Application.Json) {
handle {
call.respond("ok")
}
}
}
route("/some/plain/path", HttpMethod.Get) {
contentType(ContentType.Application.Json) {
handle {
call.respond("ok")
}
}
}
}
}
$ curl -i -XPOST localhost:3000/some/plain/path -H'Content-Type: application/xml'
HTTP/1.1 405 Method Not Allowed
Content-Length: 0
Compressing the response will result in unexpected ERROR log output after processing in the StatusPages
sample code
/compress
returns a compressed response./normal
returns a not compressed response.
fun Application.module() {
val logger = LoggerFactory.getLogger(Application::class.java)
install(Compression) {
condition {
request.uri.startsWith("/compress")
}
}
install(StatusPages) {
exception<IllegalStateException> { call, cause ->
logger.info("error! request uri: {}", call.request.uri)
call.respond(HttpStatusCode.BadRequest, "400error!!!!")
}
}
routing {
get("/compress") { throw IllegalStateException() }
get("/normal") { throw IllegalStateException() }
}
}
result(log output)
Compressing the response produces an unexpected ERROR log.
2023-02-01 18:10:33.526 [eventLoopGroupProxy-4-1] INFO i.k.server.application.Application - error! request uri: /normal
2023-02-01 18:10:41.711 [eventLoopGroupProxy-4-1] INFO i.k.server.application.Application - error! request uri: /compress
2023-02-01 18:10:41.736 [eventLoopGroupProxy-4-1] ERROR ktor.application - 400 Bad Request: GET - /compress
Cause
Could the following code be the cause? The response is not changed to sent.
The exception is resent because the response has not been sent.
https://github.com/ktorio/ktor/blob/f97c1a3432041471c1ae3eed2ecc9047c8ad44a7/ktor-server/ktor-server-core/jvmAndNix/src/io/ktor/server/application/hooks/CommonHooks.kt#L40-L49
Javadoc for Resources.kt cannot be compiled
The Javadoc for Resources.kt shows the following example, which is wrong (the Users data class must have at least one primary constructor parameter). So the best fix would probably be to switch out the data classes with regular classes.
I wouldn't mind opening a pull request btw.
/**
* Adds support for type-safe routing using [ResourcesCore].
*
* Example:
* ```kotlin
* @Serializable
* @Resource("/users")
* data class Users {
* @Serializable
* @Resource("/{id}")
* data class ById(val parent: Users = Users(), val id: Long)
*
* @Serializable
* @Resource("/add")
* data class Add(val parent: Users = Users(), val name: String)
* }
Shared
kotlinx.serialization.SerializationException is lost for the classes that have generic type parameters
Consider this class:
@Serializable
class Foo<T>(val f: Float, val t: T?)
And this test that uses kotlinx.serialization.json.Json { allowSpecialFloatingPointValues = false }
(other test infrastructure taken from Ktor's JsonClientKotlinxSerializationTest
):
@Test fun testGenericFloat(): Unit = testWithEngine(CIO) {
configureClient()
test { client ->
val result = client.post {
url(path = "/echo", port = serverPort)
contentType(defaultContentType)
setBody(Foo<String>(Float.NEGATIVE_INFINITY, "f"))
}.body<Foo<String>>()
}
}
The problem is following: normally, Json.encodeToString
can throw arbitrary SerializationException
to indicate problems with input data - e.g. invalid floating point value. It is not the only exception we can throw, but it is very simple to be illustrative.
However, Ktor apparently swallows this exception in this point: https://github.com/ktorio/ktor/blob/bad1c2da472646318a402342a249cc6300bc0940/ktor-shared/ktor-serialization/ktor-serialization-kotlinx/common/src/io/ktor/serialization/kotlinx/KotlinxSerializationBase.kt#L29
There are several problems with code:
- Ktor assumes that only
SerializationException
we can get is 'serializer not found', but it is not true. - Exception message or stack trace is lost.
- Instead, we proceed to do
guessSerializer
(https://github.com/ktorio/ktor/blob/bad1c2da472646318a402342a249cc6300bc0940/ktor-shared/ktor-serialization/ktor-serialization-kotlinx/common/src/io/ktor/serialization/kotlinx/KotlinxSerializationBase.kt#L34) which don't support generic classes. As a result, there is a cryptic message after running the test:Serializer for class 'Foo' is not found. Mark the class as @Serializable or provide the serializer explicitly.
, while it is clear thatFoo
is@Serializable
.
Given that mentioned https://github.com/Kotlin/kotlinx.serialization/issues/1163 is already fixed, I suggest rewriting this place
2.2.3
released 1st February 2023
Client
ContentNegotiation: "Skipping because the type is ignored" log message is unclear
What would it mean
i.k.s.p.c.ContentNegotiation - Skipping because the type is ignored.
This message is unclear
IU-223.8214.52, JRE 17.0.5+1-b653.23x64 JetBrains s.r.o., OS Windows 10(amd64) v10.0 , screens 1920.0x1080.0, 1920.0x1080.0
FileStorage throws java.io.FileNotFoundException (File name too long) when request path is long
REQUEST https://api.github.com/repos/Example/Repo/compare/eap...GL-1412-folder-with-multi-repos-in-it-the-you-are-offline-footer-stays-+-doesnt failed with exception: java.io.FileNotFoundException: /Users/username/Library/Caches/AppName/68747470733a2f2f6170692e6769746875622e636f6d2f7265706f732f4769744c6976654170702f636f72652f636f6d706172652f6561702e2e2e474c2d313431322d666f6c6465722d776974682d6d756c74692d7265706f732d696e2d69742d7468652d796f752d6172652d6f66666c696e652d666f6f7465722d73746179732d2b2d646f65736e74 (File name too long)
Environment
- Kotlin version: 1.8
- Library version: 1.4.1
- Kotlin platforms: JVM
- Gradle version: 7.4.1
- OS Version: macOS 12.5
HttpRequestRetry retries on FileNotFoundException thrown by FileStorage
With a configuration that includes HttpCache and HttpRequestRetry plugins such as:
HttpClient {
expectSuccess = true
install(HttpCache) { privateStorage(FileStorage(Files.createDirectories(Paths.get(dirs.cacheDir)).toFile())) }
install(ContentNegotiation) { json(json) }
Logging {
level = LogLevel.INFO
}
install(HttpRequestRetry) {
maxRetries = 5
}
Exceptions thrown by HttpCache such as FileNotFoundException (see KTOR-5443) cause HttpRequestRetry to retry the request and hit the cache again. I would expect the behavior to be that any exceptions thrown accessing the cache do not fail the request but instead continue the request as if there was a cache miss.
Add Client Plugins Trace Logging
Docs
Remove Ktor samples that duplicate existing samples in docs
- Remove from https://ktor.io/learn/
- Redeploy the site
- Update TeamCity configuration
- Remove from the repo: https://github.com/ktorio/ktor-samples
Generator
Cannot compile a project generated with the Exposed plugin
To reproduce, generate a project with the Exposed plugin. As a result, the build fails with the Overload resolution ambiguity
error:
public final val h2_version: String defined in Build_gradle
public final val h2_version: String defined in Build_gradle
Migration breaks Gson import
To reproduce, create a new project with Ktor 2.1.3 through the wizard with the Gson
plugin and run "Migrate Project to Latest Ktor Version...".
As a result, the correct import statement import io.ktor.serialization.gson.*
is replaced with the incorrect one import io.ktor.serialization.kotlinx.json.gson.*
in the plugins/Serialization.kt
file.
IntelliJ IDEA Plugin
`parent` params from nested resources are erroneously represented in OpenAPI
Example:
@Resource("/v1")
class V1Api {
@Resource("/reports")
class Reports(val parent: V1Api) {
@Resource("/")
class New(val parent: Reports)
@Resource("{id}")
class Id(val parent: Reports, val id: Int) {
@Resource("/timeline")
class Timeline(val parent: Id)
}
}
}
routing {
post<V1Api.Reports.New> {
call.respondText("hello")
}
}
Resulting schema contains:
post:
description: ""
parameters:
- name: "parent"
in: "query"
required: true
schema:
type: "object"
- name: "parent"
in: "query"
required: true
schema:
type: "object"
While parent
fields should not be considered as query parameters, but as a "technical" references to the parent class in the code (see https://ktor.io/docs/type-safe-routing.html#resource_nested)
Server
Multipart File doesn't upload whole file, throws "Unexpected EOF: expected 4096 more bytes" for larger files
Steps to reproduce:
- check out the reproducer project. It contains a simple upload sample and Gradle
run
task that constrains memory to 128mb. - Create a large file to upload (e.g. on macOS with
mkfile -n 2g upload.bin
) - Navigate to
http://0.0.0.0:8080/upload
and upload the file
The significant part seems to be UploadRoute
's stream-copying:
it.streamProvider().use { partstream ->
targetFile.outputStream().use { os ->
partstream.copyTo(os, 8192)
}
}
It results in the following:
2021-11-21 20:49:35.681 [eventLoopGroupProxy-4-1] ERROR ktor.application - Unhandled exception caught for CoroutineName(call-handler)
kotlinx.coroutines.channels.ClosedReceiveChannelException: Unexpected EOF: expected 4096 more bytes
at io.ktor.utils.io.ByteBufferChannel.readFullySuspend(ByteBufferChannel.kt:573)
at io.ktor.utils.io.ByteBufferChannel.readFully(ByteBufferChannel.kt:565)
at io.ktor.utils.io.ByteBufferChannel.readPacketSuspend(ByteBufferChannel.kt:781)
at io.ktor.utils.io.ByteBufferChannel.readPacket$suspendImpl(ByteBufferChannel.kt:767)
at io.ktor.utils.io.ByteBufferChannel.readPacket(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteReadChannelKt.readPacket(ByteReadChannel.kt:198)
at io.ktor.http.cio.MultipartKt$parseMultipart$1.invokeSuspend(Multipart.kt:342)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:205)
at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2138)
at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:887)
at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:483)
at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:678)
at io.ktor.utils.io.ByteBufferChannel.readAvailable(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteBufferChannel.readAvailableSuspend(ByteBufferChannel.kt:722)
at io.ktor.utils.io.ByteBufferChannel.access$readAvailableSuspend(ByteBufferChannel.kt:24)
at io.ktor.utils.io.ByteBufferChannel$readAvailableSuspend$2.invokeSuspend(ByteBufferChannel.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:260)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:831)
My two-gigabyte sample files have only arrived in part to /tmp
, as well.
DropwizardMetricsPlugin logs status code incorrectly when is used together with StatusPages plugin
Event Routing.RoutingCallFinished
happens before hook CallFailed
.
Because of this all the responses that were handled by the StatusPages plugin has incorrect metrics status 0.
DropwizardMetrics.kt
on(MonitoringEvent(Routing.RoutingCallFinished)) { call ->
val routingMetrics = call.attributes.take(routingMetricsKey)
val status = call.response.status()?.value ?: 0
val statusMeter =
pluginConfig.registry.meter(name(pluginConfig.baseName, routingMetrics.name, status.toString()))
statusMeter.mark()
routingMetrics.context.stop()
}
StatusPages.kt
on(CallFailed) { call, cause ->
if (call.attributes.contains(statusPageMarker)) return@on
LOGGER.trace("Call ${call.request.uri} failed with cause $cause")
val handler = findHandlerByValue(cause)
if (handler == null) {
LOGGER.trace("No handler found for exception: $cause for call ${call.request.uri}")
throw cause
}
call.attributes.put(statusPageMarker, Unit)
call.application.mdcProvider.withMDCBlock(call) {
LOGGER.trace("Executing $handler for exception $cause for call ${call.request.uri}")
handler(call, cause)
}
}
In the documentation there is an example that uses ResponseSent hook which seems to me more suitable for such case https://ktor.io/docs/events.html#custom-events
Is there any workaround, or should the DropwizardMetricsPlugin be modified to subscribe to another hook?
I also saw a discussion here to back up this idea https://github.com/ktorio/ktor/pull/2792#discussion_r791846131 however this branch wasn't merged to main branch afaics.
Log HTTP request time
I installed CallLogging on my Ktor server and it works great to keep track of what's going on!
I was wondering if there is a way to also log the ms it took to process the request, as of now it looks like this:
2020-11-05 14:12:04.248 [ktor-jetty-8080-5] INFO Application - 200 OK: GET - /users/23
Netty: Unable to set the `tcpKeepAlive`
Hi, team!
Problem:
I'm trying to set tcp keep-alive settings for our ktor netty server:
embeddedServer(Netty, port = Configuration.appPort, configure = {
tcpKeepAlive = true
...
}
...
)
However, netty warns me that SO_KEEPALIVE
is an unknown option: {"timestamp":"2022-12-21 17:27:23.982","level":"WARN","thread":"main","logger":"io.netty.bootstrap.ServerBootstrap","message":"Unknown channel option 'SO_KEEPALIVE' for channel '[id: 0x0bdc9e8b]'","context":"default"}
Workaround:
The problem could be solved with this workaround:
embeddedServer(Netty, port = Configuration.appPort, configure = {
configureBootstrap = {
childOption(ChannelOption.SO_KEEPALIVE, true)
}
...
}
...
)
Possible fix:
So, it looks like, you are setting SO_KEEPALIVE
option in a wrong way:
if (configuration.tcpKeepAlive) {
option(NioChannelOption.SO_KEEPALIVE, true)
}
Details:
implementation("io.ktor:ktor-server-netty:2.0.1")
implementation("io.netty:netty-transport-native-epoll:4.1.77.Final:linux-x86_64")
implementation("io.netty:netty-transport-native-kqueue:4.1.77.Final:osx-x86_64")
HOCON: CLI parameters don't override custom array properties since 2.1.0
Even after fixing https://youtrack.jetbrains.com/issue/KTOR-5000 CLI parameters not overriding custom array properties.
application.conf
example:
ktor {
deployment {
port = 8085
}
}
some {
custom {
arrProp = [
pumpurum,
// ololo,
]
}
}
Command example:
java -jar build/libs/app.jar -P:some.custom.arrProp.0=azaza -P:some.custom.arrProp.1=ololo -P:some.custom.arrProp.2=ururu
Expected bevaviour:
val n = config.config("some.custom").property("arrProp").getList()
// n is ["azaza", "ololo", "ururu"] (from command line)
Actual behaviour:
val n = config.config("some.custom").property("arrProp").getList()
// n is ["pumpurum"] (default value from file)
Other
Support `call.receive` (request body) in OpenAPI generator
Example. When user calls
val report = call.receive<Report>()
There should be the following entry in OpenAPI:
requestBody:
content:
'*/*':
schema:
$ref: "#/components/schemas/Report"
required: true
Make OAuth2 functionality multiplatform
Summary
OAuth2 functionality is currently JVM-only code under ktor-features/ktor-auth/
.
But I'm wondering if this should be a common client feature and be ported to ktor-client-features/ktor-client-auth/
Environment
Ktor version: 1.4.1
Subsystem: Client
Engine: OkHttp
Details
Perhaps it is my lack of understanding or perhaps it's already a tech-debt item, but I don't think there is anything JVM-specific in the implementatoin of the OAuth2 authorization flows in OAuth2.kt
.
On Android/JVM, I successfully used the verifyWithOAuth2
method to retrieve an access token. However, my ultimate goal is doing this on Windows / mingw64.
OAuth2 functionality currently is JVM-only code under
ktor-features/ktor-auth/jvm/src/io/ktor/auth
However, this seems like it should be a common client feature and therefore be ported to
ktor-client-features/ktor-client-auth/common
2.2.2
released 4th January 2023
Client
iOS unit test deadlocks with DarwinClientEngine
HttpRequestRetry: Memory leak of coroutines objects when using the plugin
We are experiencing a memory leak in our Ktor application that is tied to the HttpRequestRetry Plugin. We have an endpoint which calls a service using an HttpClient with the CIO engine (I also switch to the Java engine and saw the same behavior) when hit. On our processes that receive higher levels of traffic we saw the heap grow excessively over time without any ability to recover the heap space. After extensive testing and many many heap dumps, I realized that there were objects tied to the HttpRequestRetry Plugin. Once, I removed the plugin, all issues went away.
The objects that I am seeing in the heap are seen in the screen shot below. The kotlinx.coroutines.InvokeOnCompletion, kotlinx.coroutines.NodeList, kotlinx.coroutines.ChildHandleNode and other coroutine objects become the top objects in the heap by size. When I remove the plugin, these objects are no longer a major presence in the heap.
{width=70%}
Our configuration of the plugin is basic
HttpClient(CIO) {
install(ContentNegotiation) {
json()
}
install(Logging) {
level = LogLevel.valueOf(System.getenv("HTTP_CLIENT_LOGGING_LEVEL"))
}
defaultRequest {
headers {
ABSTracing.id?.let { append(Constants.HEADERS.ABS_TRACING_ID_HEADER, it) }
}
}
install(HttpRequestRetry) {
retryOnServerErrors(maxRetries = 3)
retryOnException(maxRetries = 3)
exponentialDelay()
modifyRequest {
request.headers["x-retry-count"] = retryCount.toString()
}
}
}
}
Gzip encoding: IllegalStateException: Expected 112, actual 113
This is a follow up from https://youtrack.jetbrains.com/issue/KTOR-4653.
Commit 4e4d0e11 introduced checks that response.body.length == response.headers["content-length"]
in the non-gzip case:
val contentLength = response.contentLength()
val contentEncoding = response.headers[HttpHeaders.ContentEncoding]
if (contentEncoding == null && contentLength != null && contentLength > 0) {
check(bytes.size == contentLength.toInt()) { "Expected $contentLength, actual ${bytes.size}" }
}
In a browser context though, the "content-encoding"
header is not exposed to JS for security reasons. From MDN docs:
Only the CORS-safelisted response headers are exposed by default. For clients to be able to access other headers,
the server must list them using the Access-Control-Expose-Headers header.
Relying on content-encoding
is a risky assumption to make and breaks in some cases. For an example, the below throws:
val client = io.ktor.client.HttpClient(Js)
val response = client.get("https://raw.githubusercontent.com/apollographql/apollo-kotlin/main/renovate.json")
// IllegalStateException: Expected 112, actual 113
println(response.body<ByteArray>())
I'm not sure what the best fix is there. Maybe just removing the check completely? Given it's not done for String
conversions, is there a specific reason to do it for ByteArray
?
Core
Allow specifying immutable in CacheControl
The CacheControl
class does not allow for the full range of valid options. For example the immutable
option which is useful for static resources.
Docs
Document how to generate OpenAPI specification for EngineMain servers
Use more clear term instead of 'authorization scope' in auth docs
https://ktor.io/docs/authentication.html#authenticate-route
for example, Protect specific routes
IntelliJ IDEA Plugin
Open API: Support request headers in OpenAPI generator
example:
get("/cards") {
call.request.header("id")
call.respond(HttpStatusCode.OK, "Your card is found")
}
In OpenAPI there should be the information about request header "id"
OpenAPI: support request cookies in documentation
example:
get("/cards") {
call.request.cookie("id")?.toInt()
call.respond(HttpStatusCode.OK, "Your card is found")
}
In OpenAPI there should be the information about request cookie "id" of type integer
Open API: Support a list of values as request header
I want request header with a value as a list of strings:
get("/report/{reportType}") {
val reportType = call.parameters["reportType"]
val repeatedAccountsId: List<String>? = call.request.headers.getAll("Authorisation") // Multiple values
call.respondText("This report is for $repeatedAccountsId and it contains type $reportType!")
}
I expect that Open API documentation will be like this:
/report/{reportType}:
get:
description: ""
parameters:
- name: "reportType"
in: "path"
required: true
schema:
type: "string"
- name: "repeatedAccountsId"
in: "header"
required: false
schema:
type: array
items: {
type: string
}
But I've got Open API documentation as for header with a String value:
/report/{reportType}:
get:
description: ""
parameters:
- name: "reportType"
in: "path"
required: true
schema:
type: "string"
- name: "Authorisation"
in: "header"
required: false
schema:
type: "string"
Ktor templates use translated string for declaring the dependency when using the Language Pack
Ktor install Authentication templates popup this: {width=339px}
and choose JWT or some other needed a new gradle dependency, will cause a error:
{width=70%}
the keyword 'implementation' has been translated, which is should not translated.
Server
Engine shutdown grace period and timeout are not configurable
In each of the EngineMain implementations, a shutdown hook is added with a default grace period and shutdown timeout applied. See:
- https://github.com/ktorio/ktor/blob/main/ktor-server/ktor-server-netty/jvm/src/io/ktor/server/netty/EngineMain.kt#L24
- https://github.com/ktorio/ktor/blob/0081f943b434bdd0afd82424b389629e89a89461/ktor-server/ktor-server-jetty/jvm/src/io/ktor/server/jetty/EngineMain.kt#L24
- https://github.com/ktorio/ktor/blob/0081f943b434bdd0afd82424b389629e89a89461/ktor-server/ktor-server-tomcat/jvm/src/io/ktor/server/tomcat/EngineMain.kt#L26
- https://github.com/ktorio/ktor/blob/be1e701bf693f2bb958fecb6bb29320384168a92/ktor-server/ktor-server-cio/jvmAndNix/src/io/ktor/server/cio/EngineMain.kt#L23
I need these to be configurable, so I can tune the grace period and timeout durations for my application. Because it is not configurable, I have to duplicate the EngineMain block in my application code, which isn't ideal. Specifically, I would love to be able to specify this in my application.conf file via something like ktor.shutdown.gracePeriodMs.
This is a change I would be open to contributing, but I would need guidance on what things I should change to accomplish this.
Server cannot be started with the Swagger plugin
Create a Ktor server, include Swagger plugin (version 2.2.1) and link it as provided in documentation, e.g.:
swaggerUI(path = "api", swaggerFile = "openapi/openapi.yml")
This works correctly when run in IntelliJ. Once i bundle the app into JAR using Shadow, server is unable to start.
java -server -jar app.jar
Exception in thread "main" java.lang.IllegalArgumentException: URI is not hierarchical
at java.base/java.io.File.<init>(File.java:420)
at io.ktor.server.plugins.swagger.SwaggerKt.swaggerUI(Swagger.kt:29)
at io.ktor.server.plugins.swagger.SwaggerKt.swaggerUI$default(Swagger.kt:22)
The swaggerUI method is too restrictive and cannot be called inside a route
Right now the swaggerUI
method is the extension for the Routing
class which prevents calling it inside a route, for example, to protect it with authentication.
Netty, HSTS: UnsupportedOperationException is thrown when the server responds before HSTS plugin
I installed the HSTS Plugin to my KTOR app and always, when a static file is requested, i get this Exception:
2022-12-05 15:41:40.202 [eventLoopGroupProxy-4-3] INFO ktor.application - 200 OK: GET - /static/bootstrap/dist/css/bootstrap.min.css.map
2022-12-05 15:41:40.205 [eventLoopGroupProxy-4-3] ERROR ktor.application - 200 OK: GET - /static/bootstrap/dist/css/bootstrap.min.css.map
java.lang.UnsupportedOperationException: Headers can no longer be set because response was already completed
at io.ktor.server.netty.http1.NettyHttp1ApplicationResponse$headers$1.engineAppendHeader(NettyHttp1ApplicationResponse.kt:42)
at io.ktor.server.response.ResponseHeaders.append(ResponseHeaders.kt:57)
at io.ktor.server.response.ResponseHeaders.append$default(ResponseHeaders.kt:48)
at io.ktor.server.response.ApplicationResponsePropertiesKt.header(ApplicationResponseProperties.kt:14)
at io.ktor.server.plugins.hsts.HSTSKt$HSTS$2$1.invokeSuspend(HSTS.kt:109)
at io.ktor.server.plugins.hsts.HSTSKt$HSTS$2$1.invoke(HSTS.kt)
at io.ktor.server.plugins.hsts.HSTSKt$HSTS$2$1.invoke(HSTS.kt)
at io.ktor.server.application.PluginBuilder$onCall$2.invokeSuspend(PluginBuilder.kt:90)
at io.ktor.server.application.PluginBuilder$onCall$2.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onCall$2.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invokeSuspend(PluginBuilder.kt:217)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invokeSuspend(PluginBuilder.kt:200)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invoke(PluginBuilder.kt)
at io.ktor.util.debug.ContextUtilsKt$addToContextInDebugMode$2.invokeSuspend(ContextUtils.kt:33)
at io.ktor.util.debug.ContextUtilsKt$addToContextInDebugMode$2.invoke(ContextUtils.kt)
at io.ktor.util.debug.ContextUtilsKt$addToContextInDebugMode$2.invoke(ContextUtils.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:89)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:169)
at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
at io.ktor.util.debug.ContextUtilsKt.addToContextInDebugMode(ContextUtils.kt:33)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invokeSuspend(PluginBuilder.kt:196)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invoke(PluginBuilder.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.UndispatchedCoroutine.afterResume(CoroutineContext.kt:233)
at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:288)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
It seems that the reason for that is the usage of onCall
here. I tested it with onCallRespond
instead and it seemed to work without throwing those Exceptions. Hope, someone can fix this.
Shared
Resource annotation should be MetaSerializable
Other
Regression in 2.2.1: Got EOF but at least 0 bytes were expected
I tried to update ktor from 2.1.3 to 2.2.1 in the IJ Platform, but I get the following error:
Exception in thread "main" java.io.EOFException: Got EOF but at least 0 bytes were expected
at io.ktor.utils.io.ByteBufferChannel.read$suspendImpl(ByteBufferChannel.kt:1658)
at io.ktor.utils.io.ByteBufferChannel.read(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteBufferChannel.readBlockSuspend(ByteBufferChannel.kt:1713)
at io.ktor.utils.io.ByteBufferChannel.access$readBlockSuspend(ByteBufferChannel.kt:23)
at io.ktor.utils.io.ByteBufferChannel$readBlockSuspend$1.invokeSuspend(ByteBufferChannel.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
I "fixed" it by catching EOFException
and ignoring it, but then I get:
java.io.EOFException: Failed to parse HTTP response: unexpected EOF
at io.ktor.client.engine.cio.UtilsKt$readResponse$2.invokeSuspend(utils.kt:132)
at (Coroutine boundary. ( )
at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:100)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:70)
at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:138)
at io.ktor.client.plugins.HttpRequestRetry$intercept$1.invokeSuspend(HttpRequestRetry.kt:291)
at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:61)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:147)
at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:104)
at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:100)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:70)
at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:138)
at io.ktor.client.plugins.HttpRequestRetry$intercept$1.invokeSuspend(HttpRequestRetry.kt:291)
at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:61)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:147)
at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:104)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:126)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:35)
at io.ktor.client.HttpClient.execute$ktor_client_core(HttpClient.kt:191)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:108)
at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:47)
at org.jetbrains.intellij.build.KtorKt$downloadFileToCacheLocation$$inlined$useWithScope2$1.invokeSuspend(ktor.kt:114)
So, I guess it is some kind of regression, since code was working previously. Probably the result of https://youtrack.jetbrains.com/issue/KTOR-5252 fix.
2.2.1
released 8th December 2022
Other
Hotfix: update atomicfu version
Make atomicfu version compatible with kotlin 1.7.*, so the processing worked correctly
2.2.0
released 7th December 2022
Build System Plugin
Enable GraalVM Metadata Repository in Gradle plugin
After we publish Ktor-related GraalVM metadata to the GraalVM Reachability Metadata Repository (ticket KTOR-5179), users will not have to generate Ktor-related metadata by themselves or configure GraalVM plugin in order to build GraalVM native image.
Add BOM file to the dependencies provided by the Gradle plugin
Client
Retry and timeout client plugins don't work together
I have both HttpRequestRetry
and HttpTimeout
installed in my client, using the Apache engine. However, when I make a request that times out, the request is not retried.
Client configuration looks like this (the timeouts are short for testing)
install(HttpRequestRetry) {
retryOnExceptionOrServerErrors(maxRetries = maxRetries)
exponentialDelay()
this.modifyRequest { println("Retry") }
}
install(HttpTimeout) {
this.connectTimeoutMillis = 40
this.requestTimeoutMillis = 80
this.socketTimeoutMillis = 80
}
When the request fails, this gets thrown. This is all that's in the logs, there's no previous throw or Retry
messages (this is with debug logging).
io.ktor.client.plugins.HttpRequestTimeoutException: Request timeout has expired [url=***, request_timeout=40 ms]
at io.ktor.client.plugins.HttpTimeout$Plugin$install$1$1$killer$1.invokeSuspend(HttpTimeout.kt:163)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Darwin: response is never returned when usePreconfiguredSession is used
Passing a custom NSURLSession to HttpClient's engine usePreconfiguredSession()
does not work. Installing Logging and printing everything with LogLevel.ALL prints headers and always empty body. The failure is quite hard to figure, but the call is never made.
HttpClient(Darwin) {
engine {
val config = NSURLSessionConfiguration.defaultSessionConfiguration()
val session = NSURLSession.sessionWithConfiguration(config)
usePreconfiguredSession(session)
}
}
@rustam.siniukov confirmed that this API is broken on this Slack thread.
This is happening with the latest Ktor's release: 2.1.3
Logging: the plugin instantiates the default logger even when a custom one is provided
I'm getting an error while trying to set a custom Logger for Ktor:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Program%20Files/JetBrains/IntelliJ%20IDEA%20Community%20Edition/plugins/maven/lib/maven36-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Program%20Files/JetBrains/IntelliJ%20IDEA%20Community%20Edition/lib/3rd-party-rt.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.jetbrains.idea.maven.server.Maven3WrapperSl4LoggerFactory]
java.lang.LinkageError: loader constraint violation: when resolving method "org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;" the class loader com.intellij.ide.plugins.cl.PluginClassLoader @2fc90a94 (instance of com.intellij.ide.plugins.cl.PluginClassLoader, child of 'bootstrap') of the current class, org/slf4j/LoggerFactory, and the class loader com.intellij.ide.plugins.cl.PluginClassLoader @55b14fad (instance of com.intellij.ide.plugins.cl.PluginClassLoader, child of 'bootstrap') for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at io.ktor.client.plugins.logging.LoggerJvmKt$DEFAULT$1.<init>(LoggerJvm.kt:12)
at io.ktor.client.plugins.logging.LoggerJvmKt.getDEFAULT(LoggerJvm.kt:11)
at io.ktor.client.plugins.logging.Logging$Config.<init>(Logging.kt:45)
at io.ktor.client.plugins.logging.Logging$Companion.prepare(Logging.kt:216)
at io.ktor.client.plugins.logging.Logging$Companion.prepare(Logging.kt:212)
at io.ktor.client.HttpClientConfig$install$3.invoke(HttpClientConfig.kt:84)
at io.ktor.client.HttpClientConfig$install$3.invoke(HttpClientConfig.kt:81)
at io.ktor.client.HttpClientConfig.install(HttpClientConfig.kt:104)
This is the code for the client:
HttpClient(CIO) {
install(Logging) {
logger = object : Logger {
override fun log(message: String) {
println(message)
}
}
}
}
You can see that a new instance of config is created in prepare()
:
public companion object : HttpClientPlugin<Config, Logging> {
override val key: AttributeKey<Logging> = AttributeKey("ClientLogging")
override fun prepare(block: Config.() -> Unit): Logging {
val config = Config().apply(block)
return Logging(config.logger, config.level, config.filters)
}
override fun install(plugin: Logging, scope: HttpClient) {
plugin.setupRequestLogging(scope)
plugin.setupResponseLogging(scope)
}
}
Which creates a new instance of the default logger, even if I provided a custom one:
@KtorDsl
public class Config {
/**
* filters
*/
internal var filters = mutableListOf<(HttpRequestBuilder) -> Boolean>()
/**
* Specifies a [Logger] instance.
*/
public var logger: Logger = Logger.DEFAULT
/**
* Specifies the log the logging level.
*/
public var level: LogLevel = LogLevel.HEADERS
/**
* Allows you to filter log messages for calls matching a [predicate].
*/
public fun filter(predicate: (HttpRequestBuilder) -> Boolean) {
filters.add(predicate)
}
}
Could anyone suggest a workaround, until this is fixed?
See:
Java client engine doesn't handle HttpTimeout.INFINITE_TIMEOUT_MS properly
val client = HttpClient(Java) {
install(HttpTimeout) {
connectTimeoutMillis = HttpTimeout.INFINITE_TIMEOUT_MS
socketTimeoutMillis = HttpTimeout.INFINITE_TIMEOUT_MS
requestTimeoutMillis = HttpTimeout.INFINITE_TIMEOUT_MS
}
}
casues
java.lang.ArithmeticException: long overflow
at java.base/java.lang.Math.multiplyExact(Math.java:946)
at java.base/java.lang.Math.multiplyExact(Math.java:922)
at java.base/java.time.Instant.toEpochMilli(Instant.java:1236)
at java.base/java.time.Instant.until(Instant.java:1149)
at java.net.http/jdk.internal.net.http.HttpClientImpl.purgeTimeoutsAndReturnNextDeadline(HttpClientImpl.java:1222)
at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:877)
New plugins API for client
Persistent Client HttpCache
Currently the HttpCache is implemented as in-memory only. Accordingly using it in an iOS/Android multiplatform app, cache entries are lost when closing the app. It would be nice to have a file-based persistent cache like OkHttp (https://square.github.io/okhttp/4.x/okhttp/okhttp3/-cache/).
Support native windows HTTP client
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/671
To create native windows applications could be useful to have native HTTP client which supports using Windows certificates store for SSL connections, proxy settings configuration on the system level, etc.
Microsoft recommends using WinHTTP for server-side applications and services. WinHTTP is used in .NET windows http client so it's a good candidate to use it on Windows. WinHTTP could work in sync and async mode, which is recommended to use and async calls could be wrapped by using Kotlin coroutines.
Websockets, Darwin: trusting a certificate via `handleChallenge` doesn't work for Websockets connections
We have dev environment with a self-signed certificate, we use ktor (2.1.3) for REST and web sockets,
REST endpoints work pretty well, but web sockets crash with errors:
Task <A986E3F4-E43F-4909-A663-5B1CBD50FFEF>.<1> finished with error [-1202] Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “walp.kube” which could put your confidential information at risk." UserInfo={NSErrorFailingURLStringKey=wss://walp.kube/ws, NSErrorFailingURLKey=wss://walp.kube/ws, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalWebSocketTask <A986E3F4-E43F-4909-A663-5B1CBD50FFEF>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalWebSocketTask <A986E3F4-E43F-4909-A663-5B1CBD50FFEF>.<1>, NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “walp.kube” which could put your confidential information at risk.}
HttpClient: REQUEST wss://walp.kube/ws failed with exception: kotlinx.coroutines.JobCancellationException: Parent job is Cancelling; job=JobImpl{Cancelling}@2f08970
Exception doesn't match @Throws-specified class list and thus isn't propagated from Kotlin to Objective-C/Swift as NSError.
It is considered unexpected and unhandled instead. Program will be terminated.
Uncaught Kotlin exception: io.ktor.client.engine.darwin.DarwinHttpRequestException: Exception in http request: Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “walp.kube” which could put your confidential information at risk." UserInfo={NSErrorFailingURLStringKey=wss://walp.kube/ws, NSErrorFailingURLKey=wss://walp.kube/ws, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “walp.kube” which could put your confidential information at risk.}
at 0 WalletApi 0x116af7767 kfun:io.ktor.client.engine.darwin.DarwinHttpRequestException#<init>(platform.Foundation.NSError){} + 167
at 1 WalletApi 0x116afcdaf kfun:io.ktor.client.engine.darwin.$receiveMessage$lambda-0$FUNCTION_REFERENCE$76.$<bridge-UNNNN>invoke(-1:0;-1:1){}#internal + 223
at 2 WalletApi 0x116c9fa1f ___696f2e6b746f723a6b746f722d636c69656e742d64617277696e_knbridge45_block_invoke + 507
at 3 Foundation 0x1807ec16f __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 15
at 4 Foundation 0x1807ec04b -[NSBlockOperation main] + 99
at 5 Foundation 0x1807ef16f __NSOPERATION_IS_INVOKING_MAIN__ + 19
at 6 Foundation 0x1807eb1cf -[NSOperation start] + 759
at 7 Foundation 0x1807efb07 __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 19
at 8 Foundation 0x1807ef60f __NSOQSchedule_f + 179
at 9 libdispatch.dylib 0x106310f9b _dispatch_block_async_invoke2 + 103
at 10 libdispatch.dylib 0x106301b93 _dispatch_client_callout + 15
at 11 libdispatch.dylib 0x10630489b _dispatch_continuation_pop + 555
at 12 libdispatch.dylib 0x106303cff _dispatch_async_redirect_invoke + 695
at 13 libdispatch.dylib 0x106313feb _dispatch_root_queue_drain + 439
at 14 libdispatch.dylib 0x106314ab7 _dispatch_worker_thread2 + 187
at 15 libsystem_pthread.dylib 0x1cc0adb03 _pthread_wqthread + 223
at 16 libsystem_pthread.dylib 0x1cc0ac903 start_wqthread + 7
dyld4 config: DYLD_ROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot DYLD_LIBRARY_PATH=/Users/s.iastremskyi/Library/Developer/Xcode/DerivedData/CEXWallet-frzuyzmdwpwcxcbdtqjosexjngpm/Build/Products/Debug-iphonesimulator:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/introspection DYLD_INSERT_LIBRARIES=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/usr/lib/libBacktraceRecording.dylib:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/usr/lib/libMainThreadChecker.dylib:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib DYLD_FRAMEWORK_PATH=/Users/s.iastremskyi/Library/Developer/Xcode/DerivedData/CEXWallet-frzuyzmdwpwcxcbdtqjosexjngpm/Build/Products/Debug-iphonesimulator:/Users/s.iastremskyi/Library/Developer/Xcode/DerivedData/CEXWallet-frzuyzmdwpwcxcbdtqjosexjngpm/Build/Products/Debug-iphonesimulator/PackageFrameworks
CoreSimulator 857.7 - Device: iPhone 13 Pro Max (6E9732C3-9A52-46CD-BDA9-9C1F5538EA0B) - Runtime: iOS 15.5 (19F70) - DeviceType: iPhone 13 Pro Max
Client (iOS) configured for trust skip as well:
actual val clientEngine: HttpClientEngine = Darwin.create {
handleChallenge { session, task, challenge, completionHandler ->
challenge.protectionSpace.serverTrust?.let {
if (challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodServerTrust) {
val credential = NSURLCredential.credentialForTrust(it)
completionHandler(NSURLSessionAuthChallengeUseCredential, credential)
}
}
}
}
Jackson converter: Support requests with Content-Length header
Hello,
With Ktor 2.0 release there was a change in default client behavior when handling JSON - changed from setting Content-Length
to a Transfer-Encoding: chunked
header on the request. Reverting to old behavior is not trivial - we would either need to manually compute the length of the request and explicitly set it on the headers OR write custom Jackson converter that uses TextContent
instead of OutputStreamContent
. It would be great if end users could select the appropriate behavior without any custom code.
---
Example code included below for reference (introspection query is just a String constant).
v1.6 code
HttpClient(engineFactory = Apache) {
install(HttpTimeout) {
connectTimeoutMillis = connectTimeout
requestTimeoutMillis = readTimeout
}
install(feature = JsonFeature)
}.use { client ->
runBlocking {
val introspectionResult = try {
client.post<Map<String, Any?>> {
url(endpoint)
contentType(ContentType.Application.Json)
httpHeaders.forEach { (name, value) ->
header(name, value)
}
body = mapOf(
"query" to INTROSPECTION_QUERY,
"operationName" to "IntrospectionQuery"
)
}
} catch (e: Throwable) {
when (e) {
is ClientRequestException, is HttpRequestTimeoutException, is UnknownHostException -> throw e
else -> throw RuntimeException("Unable to run introspection query against the specified endpoint=$endpoint", e)
}
}
}
v2.0 code
HttpClient(engineFactory = Apache) {
install(HttpTimeout) {
connectTimeoutMillis = connectTimeout
requestTimeoutMillis = readTimeout
}
install(ContentNegotiation) {
jackson()
}
}.use { client ->
runBlocking {
val introspectionResult = try {
client.post {
url(endpoint)
contentType(ContentType.Application.Json)
httpHeaders.forEach { (name, value) ->
header(name, value)
}
setBody(
mapOf(
"query" to INTROSPECTION_QUERY,
"operationName" to "IntrospectionQuery"
)
)
expectSuccess = true
}.body<Map<String, Any?>>()
} catch (e: Throwable) {
when (e) {
is ClientRequestException, is HttpRequestTimeoutException, is UnknownHostException -> throw e
else -> throw RuntimeException("Unable to run introspection query against the specified endpoint=$endpoint", e)
}
}
}
Core
The beginning character of encodedPath field(Url class) is wrong when relative path
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1123
Ktor Version
1.1.3
Ktor Engine Used(client or server and name)
server (Netty)
JVM Version, Operating System and Relevant Context
java 1.8.0_201
MacOS 10.14.4
Feedback
When I set relative path to Url class constructor, the beggining slash is toggled like below.
Url(“/hoge“).encodedPath=“hoge”
Url(“hoge“).encodedPath=“/hoge”
I think the beggining slash is retained like below.
Url(“/hoge“).encodedPath=“/hoge”
Url(“hoge“).encodedPath=“hoge”
parseAuthorizationHeader throws ParseException on header value with multiple challenges
Hello, I was trying to use KTOR Client 2.1.3 with the auth
plugin for OAuth2 bearer authentication with a service hosted on MS Azure. When the first request is made to the API the plugin tries to parse the WWW-authenticate header from the response in order to pick a provider, but it fails, because the value contains multiple challenges:
Basic realm="", Bearer authorization_uri="https://login.microsoftonline.com/TENANT/oauth2/authorize"
If the Mozilla Developer portal is right, this should be a valid value:
// Challenges specified in single header
WWW-Authenticate: challenge1, ..., challengeN
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/WWW-Authenticate
The method parseAuthorizationHeader
however does not seem to be prepared to handle multiple challenges, so it fails with
io.ktor.http.parsing.ParseException: Expected `=` after parameter key 'Bearer': Basic realm="", Bearer authorization_uri="https://example.com/oauth2/authorize"
The following test added to io/ktor/tests/http/AuthHeaderParseTest.kt
exposes the bug
@Test
fun testMultipleChallenges() {
val header = """
Basic realm="", Bearer authorization_uri="https://example.com/oauth2/authorize"
""".trimIndent()
val result = parseAuthorizationHeader(header)
assertTrue(result is HttpAuthHeader.Parameterized)
}
NativePRNGNonBlocking is not found, fallback to SHA1PRNG
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/964
Ktor Version
1.1.2, 1.1.3
Ktor Engine Used(client or server and name)
netty server
JVM Version, Operating System and Relevant Context
JRE 11, Windows 10
Feedback
I get the following warning:
io.ktor.util.random - NativePRNGNonBlocking is not found, fallback to SHA1PRNG
Is there anything I can do to fix that?
Remove check for internal class in Select
I want to use this c code with Kotlin. Fortunately, Ktor provides a select api:
val selector = SelectorManager(coroutineScope!!.coroutineContext)
selector.select(selectable = object : Selectable {
override val descriptor: Int = socketFileDescriptor
}, SelectInterest.READ)
Given C code
fd_set input_mask;
int sock = PQsocket(conn);
FD_ZERO(&input_mask);
FD_SET(sock, &input_mask);
select(sock + 1, &input_mask, NULL, NULL, NULL)
But the currenct Kotlin code fails, because the parameter selectable
is expected to be an internal class.
https://github.com/ktorio/ktor/blob/24d91c337fe9874ca1b5235ecdb2dbfa52cf11a9/ktor-network/nix/src/io/ktor/network/selector/WorkerSelectorManager.kt#L30
Solution: Remove this check
Docs
Create the migration guide for updating a Ktor project from 2.0.0 to 2.2.0
- Cookies: append
- host/port -> serverHost/serverPort
- https://github.com/ktorio/ktor/pull/3258
- publicStorage -> publicStorage()
- client custom plugins
- new memory model is enabled by default
Documentation doens't match the convert{} extensions in 2.1.0
Ktor install() function errors when installing Mustache integration
After reading the docutmentation on using Mustache with Ktor as seen here https://ktor.io/docs/mustache.html . I checked again and it appears that this extends to any time the install() function is called.
Site.kt
package xyz.qixt
// General deps
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.application.*
import io.ktor.features.*
import io.ktor.response.*
import io.ktor.routing.*
// Frontend imports
import com.github.mustachejava.*
import io.ktor.mustache.*
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun main(args: Array<String>) {
embeddedServer(Netty, port = 8080){}.start()
}
fun Application.pages() {
install(Mustache) {
MustacheFactory = DefaultMustacheFactory("views/mustache")
}
routing {
get ("/"){
val CurrentUser = User("Will")
call.respond(MustacheContent("index.hbs", mapOf("user" to CurrentUser)))
}
}
}
data class User(val name: String)
build.gradle.kts
import org.jetbrains.kotlin.gradle.dsl.Coroutines
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
val logback_version: String by project
val ktor_version: String by project
val kotlin_version: String by project
plugins {
application
kotlin("jvm") version "1.4.30"
}
group = "xyz.qixt"
version = "0.0.1-SNAPSHOT"
application {
//mainClassName = "io.ktor.server.netty.EngineMain"
mainClassName = "xyz.qixt.SiteKt"
}
repositories {
mavenLocal()
jcenter()
maven { url = uri("https://kotlin.bintray.com/ktor") }
maven { url = uri("https://kotlin.bintray.com/kotlin-js-wrappers") }
}
dependencies {
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version")
implementation("io.ktor:ktor-server-netty:$ktor_version")
implementation("ch.qos.logback:logback-classic:$logback_version")
implementation("io.ktor:ktor-server-core:$ktor_version")
implementation("io.ktor:ktor-html-builder:$ktor_version")
implementation("org.jetbrains:kotlin-css-jvm:1.0.0-pre.31-kotlin-1.2.41")
implementation("io.ktor:ktor-mustache:$ktor_version")
implementation("io.ktor:ktor-server-host-common:$ktor_version")
testImplementation("io.ktor:ktor-server-tests:$ktor_version")
}
kotlin.sourceSets["main"].kotlin.srcDirs("src")
kotlin.sourceSets["test"].kotlin.srcDirs("test")
sourceSets["main"].resources.srcDirs("resources")
sourceSets["test"].resources.srcDirs("resources")
output/error
src/Site.kt: (28, 5): Not enough information to infer type variable B
src/Site.kt: (28, 13): Unresolved reference: Mustache
src/Site.kt: (29, 9): Variable expected
Explain the pros and cons of each engine
After a lot of research I have only been able to find very limited info on which engine to pick for server Ktor. Until now this is all the information I have been able to gather (some of which is possibly wrong):
- For native only CIO engine is supported
- Tomcat and Jetty engines are to be used when the application is to be deployed inside the respective application servers
- Netty and CIO allow for a stand alone application, without application server
- There is an annotation that recommends using Netty if you don't know which one to use
I would very much appreciate a technical discussion (whether in the docs, or as a blog post somewhere maybe) outlining the advantages and disadvantages of each engine in terms of performance, functionalities available, limitations, stability, etc. Also it would be interesting for this discussion to include in the comparison the usage of Ktor inside Tomcat or Jetty versus as an stand alone application using Netty or CIO.
Generator
Wizard: Creating a project with sample code creates `Application.configureRouting` and `ApplicationTest` with unused imports
Creating a project with sample code and without plugins added creates `Application.configureRouting` and `ApplicationTest` with unused imports.
The code generated for Application.configureRouting():
package com.example.plugins
import io.ktor.server.routing.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.request.*
fun Application.configureRouting() {
routing {
get("/") {
call.respondText("Hello World!")
}
}
}
These imports not used:
import io.ktor.http.*
import io.ktor.server.request.*
The code generated for ApplicationTest:
package com.example
import io.ktor.server.routing.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.request.*
import kotlin.test.*
import io.ktor.server.testing.*
import com.example.plugins.*
class ApplicationTest {
}
These imports not used:
import io.ktor.server.routing.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.request.*
import kotlin.test.*
import io.ktor.server.testing.*
import com.example.plugins.*
Support YAML generation in wizard and start.ktor.io
start.ktor.io produces project with compilation errors
When selecting the following plugins initial app compilation produces errors.
Plugins:
- SwaggerUI
- DropwizardMetrics
- Resources
The project zip produced and compilation errors log is attached.
Add OpenAPI and Swagger plugins to generator
Replace the Locations plugin with a new Resources plugin
Given that starting with v2.0.0 the Resources plugin is a recommended way to implement type-safe routing, we can add it to the Ktor plugin instead of Locations.
Documentation: https://ktor.io/docs/type-safe-routing.html
Exclude 2.2.0 from Ktor generator because it's faulty
Currently wizard shows 2.2.0 version of Ktor as available while it contains errors. We must exclude it from the list and only show 2.2.1 instead.
The Ktor plugin doesn't add any code for the Metrics plugin
- Create a new project in IDEA.
- Leave 'Add sample code' enabled.
- Add the Metrics plugin.
=> Thecom/example/plugins/Monitoring.kt
file contains no code:
fun Application.configureMonitoring() {
}
Install plugin on the fly stopped working
IntelliJ IDEA 2023.1 EAP (Ultimate Edition)
Build #IU-231.3209, built on December 12, 2022
-
Create new ktor project
-
Start typing the word
install(
Expected result:
Once you complete install(<you are here>)
it will automatically pop up a list of Ktor plugins taken for your Ktor version from Markteplace
Actual:
Ktor plugins can't be found and installed on the fly
Generate application module extension even for embeddedServer (in Generator)
Upgrade to the latest Ktor version trashes kotlinx.json import statement
This has happened with prior ktor upgrades as well
What happens
The IDE notifies me 'Ktor 2.1.2 is Available. Automatic project migration is available'
I choose to upgrade the project.
I expect everything to work. Including my Applicate.configureSerialization() which includes
import io.ktor.serialization.kotlinx.json.*
But, when I restart my app, I get a compilation there, the import is replaced with import io.ktor.serialization.kotlinx.json.kotlinx.json.*
which is presumably an Automated typo for
io.ktor.serialization.kotlinx.json.*
It has been easy enough to undo that change, but it seemed time to report it.
Thanks
Gordon
IU-222.4167.29, JRE 17.0.4+7-b469.53x64 JetBrains s.r.o., OS Linux(amd64) v5.19.0-76051900-generic, screens 1600.0x900.0, 1600.0x900.0, 1920.0x1080.0
IntelliJ IDEA Plugin
Support StatusPages plugin in generator
Example:
install(StatusPages) {
registerExceptionHandlerForT()
exception<Exception> { call, cause ->
call.respond(HttpStatusCode.BAD_ACCESS, "Unknown error")
}
}
fun StatusPagesConfiguration.registerExceptionHandlerForT() {
if (cause is SomeSubclassOfT) {
call.respond(HttpStatusCode.OK, "All good")
} else {
call.respond(HttpStatusCode.ACCESS_DENIED)
}
}
route("/some/route") {
get("/first") {
throw T()
}
get("/second") {
throw SomeSubclassOfT()
}
get("/third") {
throw IndexOutOfBoundsException(10)
}
}
Then, the OpenAPI for these routes should indicate that :
/some/route/first
responds with HttpStatusCode.OK, "All good"/some/route/second
responds with HttpStatusCode.ACCESS_DENIED/some/route/third
responds with HttpStatusCode.BAD_ACCESS, "Unknown error"
Support type highlighting of Yaml files for Ktor in IDE
Support properties completion in ktor's YAML config files in IDE
example: user writes "host" , IDE suggests "ktor.deployment.host" of type INT, and puts it in YAML format in proper place.
Add completion and resolve for ktor.application.modules in YAML files
Add inspections for ktor yaml config files in IDE
Inspection examples:
- property value type is incorrect
- property is duplicated
- class/method (ktor.application.modules) is not resolvable
Support YAML in IDE/editor
IJ Microservices IDE offers quite a lot of functionality for config files, such as completion, navigation, documentation viewing.
We need to make sure it is supported for Ktor.
"Create Test" action should not configure application in the created test if it's called not from `Application` extension
Having this code in Application.kt
class:
package com.example
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.routing.*
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
routing {
get("/") {}
}
}.start(wait = true)
}
When you try creating test for this get("/")
route, you'll end up with this:
package com.example;
import io.ktor.client.plugins.websocket.*
import io.ktor.client.request.*
import io.ktor.server.testing.*
import kotlin.test.Test
class ApplicationKtTest {
@Test
fun testGetEmptyRoute() = testApplication {
application {
main()
}
client.get("").apply {
TODO("Please write your test here")
}
}
}
This application { main() }
will not configure the app, instead it will start the server and never finish.
Proposed solution is to either to:
- Remove the
application { ... }
call entirely - Remove the
main()
call insideapplication
and leave a comment likePlease configure your application here
inside, like
application {
TODO("Please configure your application here")
}
The second option seems to look better.
Module Plugins are not Applied when Generating Test for Endpoint
IntelliJ IDEA extracts a module to an extension function on cancelling
IntelliJ IDEA 2022.3 RC (Ultimate Edition)
Build #IU-223.7571.123, built on November 23, 2022
KDoc parameters/tags are not parsed properly in OpenAPI generator
Ex:
/**
* Request has the following fields:
* @param name Is a name parameter
* @see [link](https://name-param.com)
*/
get("/request/{name}") {
...
}
Link will not be parsed, and param comment is ignored
EA: Throwable: Must be executed under progress indicator: com.intellij.openapi.progress.EmptyProgressIndicator@2ff16fee. Please see e.g. ProgressManager.runProcess()
Deduce parameter types from usage in OpenAPI in cases when they are primitive (string, number, integer, boolean)
Example:
val id = call.parameters["id"]?.toInt() // <----- id is integer here
val ok = call.parameters["ok"]?.toBoolean() // <----- ok is boolean here
val digits = call.parameters["digits"]?.toDouble() // <----- digits is number here
Support headers in responses in OpenAPI generator
ex:
get("/caba/{id}") {
call.response.headers.append("myName", "abacaba")
call.respond("Hey! $id")
}
In OpenAPI there should be the information about the header "myName":
{width=70%}
Support full schemas in OpenAPI generator
Support object types with fields for respond types and parameter types
Currently we can only deduce primitive types of responses
OpenAPI is incorrectly generated for List<T> respond types (item type is missing)
For this project: https://github.com/ktorio/ktor-documentation/tree/2.1.3/codeSnippets/snippets/tutorial-http-api the response type for the /order
request would be array
without items.
Note: changing orderStorage
from List<Order>
to MutableList<Order>
fixes the issue.
The reason behind this is that PsiType for List<X>
is PsiType: List<? extends X>
while for MutableList<X>
it would be PsiType: List<X>
. Hence, the type of item is determinable only for mutable lists.
Solution: we must think of lists same way as of mutable lists in terms of OpenAPI
"Dialog shown not from EDT" and "Read access not allowed here" exceptions when generating tests with extraction in IJ 231
Add Macros to Create Ktor Client with Engine Dependencies
Add new live templates(macro) for creating HttpClient and adding required dependencies:
Example:
// typing
ktor| // here triggers completion with `ktor-client` option
// expands to:
val client = HttpClient()
// and dependencies only if there is no dependency for client engine
implementation("io.ktor:ktor-client-apache:$version")
Generating OpenAPI by default and warning level of inspection is too in your face
Resolution:
We disable generating OpenAPI for new project
1.1. We enable generating OpenAPI for new project IF OpenAPI plugin is installed
We disable auto-update by default but still suggest this option
We change the inspection level from Warning to Information
The IDEA plugin generates code with deprecated API for StatusPages
Naked underscore as a subdomain for Website field leads to error
To reproduce fill the Website
field with the value _.example.com
while creating a new project and run it afterward via run configuration.
As a result, I get the following compile-time error:
e: /home/stexe/projects/ktor_plugin/ktor-sample1/src/main/kotlin/com/example/_/Application.kt: (1, 21): Names _, __, ___, ... can be used only in back-ticks (`_`, `__`, `___`, ...)
e: /home/stexe/projects/ktor_plugin/ktor-sample1/src/main/kotlin/com/example/_/Application.kt: (5, 20): Names _, __, ___, ... can be used only in back-ticks (`_`, `__`, `___`, ...)
e: /home/stexe/projects/ktor_plugin/ktor-sample1/src/main/kotlin/com/example/_/plugins/Routing.kt: (1, 21): Names _, __, ___, ... can be used only in back-ticks (`_`, `__`, `___`, ...)
Wizard: Creating a project with sample code creates `Application.configureRouting` with two `routing` blocks
IntelliJ IDEA 2022.1 EAP (Ultimate Edition)
Build #IU-221.3635, built on January 24, 2022
Creating a project with sample code and without plugins added creates `Application.configureRouting` with two `routing` blocks
The code generated:
fun Application.configureRouting() {
// Starting point for a Ktor app:
routing {
get("/") {
call.respondText("Hello World!")
}
}
routing {
}
}
Feature Name from Source Code is Used for Marketplace
I guess we should use Conditional Headers
instead of ConditionalHeaders
(and etc)
Exceptions at creating a ktor run configuration
build 213.5744.8
-go to any project (for example, created sample ktor project)
-go to run configurations
-press +
-select Ktor
Result: the Exception below and others. Also, I've tried to create any other run configuration(spring boot, just to try it, without any reasonable intensions ) and got also the same exception and many others. See log for details.
java.lang.Throwable: Slow operations are prohibited on EDT. See SlowOperations.assertSlowOperationsAreAllowed javadoc.
at com.intellij.openapi.diagnostic.Logger.error(Logger.java:182)
at com.intellij.util.SlowOperations.assertSlowOperationsAreAllowed(SlowOperations.java:102)
at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:784)
at com.intellij.psi.stubs.StubIndexImpl.getContainingIds(StubIndexImpl.java:514)
at com.intellij.psi.stubs.StubIndexImpl.processElements(StubIndexImpl.java:318)
at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:100)
at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:88)
at com.intellij.psi.impl.java.stubs.index.JavaFullClassNameIndex.get(JavaFullClassNameIndex.java:30)
at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.doFindClasses(JavaFileManagerImpl.java:84)
at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.findClass(JavaFileManagerImpl.java:110)
at com.intellij.psi.impl.PsiElementFinderImpl.findClass(PsiElementFinderImpl.java:40)
at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:91)
at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:69)
at com.intellij.jam.JavaLibraryUtils.lambda$getLibraryClassMap$2(JavaLibraryUtils.java:56)
at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
at java.base/java.util.concurrent.ConcurrentMap.getOrDefault(ConcurrentMap.java:88)
at com.intellij.jam.JavaLibraryUtils.hasLibraryClass(JavaLibraryUtils.java:37)
at io.ktor.initializr.intellij.run.KtorLibraryUtilKt.hasKtorLibraryOnJVM(KtorLibraryUtil.kt:34)
at io.ktor.initializr.intellij.run.KtorLibraryUtilKt.hasKtorLibrary(KtorLibraryUtil.kt:45)
at io.ktor.initializr.intellij.run.KtorRunConfigurationEditor$moduleSelector$1.isModuleAccepted(KtorRunConfigurationEditor.kt:50)
at com.intellij.execution.ui.ConfigurationModuleSelector.reset(ConfigurationModuleSelector.java:136)
at com.intellij.execution.ui.ConfigurationModuleSelector.reset(ConfigurationModuleSelector.java:123)
at io.ktor.initializr.intellij.run.KtorRunConfigurationEditor.resetEditorFrom(KtorRunConfigurationEditor.kt:129)
at io.ktor.initializr.intellij.run.KtorRunConfigurationEditor.resetEditorFrom(KtorRunConfigurationEditor.kt:38)
at com.intellij.openapi.options.SettingsEditor.lambda$resetFrom$0(SettingsEditor.java:77)
at com.intellij.openapi.options.SettingsEditor.bulkUpdate(SettingsEditor.java:85)
at com.intellij.openapi.options.SettingsEditor.resetFrom(SettingsEditor.java:75)
at com.intellij.execution.impl.ConfigurationSettingsEditor$ConfigToSettingsWrapper.resetEditorFrom(ConfigurationSettingsEditor.java:307)
at com.intellij.execution.impl.ConfigurationSettingsEditor$ConfigToSettingsWrapper.resetEditorFrom(ConfigurationSettingsEditor.java:290)
at com.intellij.openapi.options.CompositeSettingsEditor.resetEditorFrom(CompositeSettingsEditor.java:44)
at com.intellij.execution.impl.ConfigurationSettingsEditorWrapper.resetEditorFrom(ConfigurationSettingsEditorWrapper.java:129)
at com.intellij.execution.impl.ConfigurationSettingsEditorWrapper.resetEditorFrom(ConfigurationSettingsEditorWrapper.java:31)
at com.intellij.openapi.options.SettingsEditor.lambda$resetFrom$0(SettingsEditor.java:77)
at com.intellij.openapi.options.SettingsEditor.bulkUpdate(SettingsEditor.java:85)
at com.intellij.openapi.options.SettingsEditor.resetFrom(SettingsEditor.java:75)
at com.intellij.openapi.options.SettingsEditorConfigurable.reset(SettingsEditorConfigurable.java:39)
at com.intellij.execution.impl.SingleConfigurationConfigurable.reset(SingleConfigurationConfigurable.java:179)
at com.intellij.execution.impl.SingleConfigurationConfigurable.editSettings(SingleConfigurationConfigurable.java:116)
at com.intellij.execution.impl.RunConfigurable.createNewConfiguration(RunConfigurable.kt:861)
at com.intellij.execution.impl.RunConfigurable.createNewConfiguration(RunConfigurable.kt:896)
at com.intellij.execution.impl.RunConfigurable$MyToolbarAddAction$showAddPopup$popup$1.consume(RunConfigurable.kt:938)
at com.intellij.execution.impl.RunConfigurable$MyToolbarAddAction$showAddPopup$popup$1.consume(RunConfigurable.kt:909)
at com.intellij.execution.impl.NewRunConfigurationPopup$5.onChosen(NewRunConfigurationPopup.java:249)
at com.intellij.ui.popup.tree.TreePopupImpl.handleSelect(TreePopupImpl.java:357)
at com.intellij.ui.popup.tree.TreePopupImpl$MyMouseListener.mousePressed(TreePopupImpl.java:307)
at java.desktop/java.awt.AWTEventMulticaster.mousePressed(AWTEventMulticaster.java:288)
at java.desktop/java.awt.Component.processMouseEvent(Component.java:6651)
at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3345)
at com.intellij.ui.treeStructure.Tree.processMouseEvent(Tree.java:394)
at java.desktop/java.awt.Component.processEvent(Component.java:6419)
at java.desktop/java.awt.Container.processEvent(Container.java:2263)
at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5029)
at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321)
at java.desktop/java.awt.Component.dispatchEvent(Component.java:4861)
at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918)
at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4544)
at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488)
at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307)
at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2790)
at java.desktop/java.awt.Component.dispatchEvent(Component.java:4861)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:778)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95)
at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:751)
at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:749)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:748)
at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:891)
at com.intellij.ide.IdeEventQueue.dispatchMouseEvent(IdeEventQueue.java:820)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:757)
at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$6(IdeEventQueue.java:447)
at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:818)
at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$7(IdeEventQueue.java:446)
at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:805)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:498)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:117)
at java.desktop/java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:190)
at java.desktop/java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:235)
at java.desktop/java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:233)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:233)
at java.desktop/java.awt.Dialog.show(Dialog.java:1070)
at com.intellij.openapi.ui.impl.DialogWrapperPeerImpl$MyDialog.show(DialogWrapperPeerImpl.java:701)
at com.intellij.openapi.ui.impl.DialogWrapperPeerImpl.show(DialogWrapperPeerImpl.java:437)
at com.intellij.openapi.ui.DialogWrapper.doShow(DialogWrapper.java:1671)
at com.intellij.openapi.ui.DialogWrapper.show(DialogWrapper.java:1629)
at com.intellij.execution.actions.EditRunConfigurationsAction.actionPerformed(EditRunConfigurationsAction.java:30)
at com.intellij.openapi.actionSystem.ex.ActionUtil.lambda$performActionDumbAwareWithCallbacks$4(ActionUtil.java:244)
at com.intellij.openapi.actionSystem.ex.ActionUtil.performDumbAwareWithCallbacks(ActionUtil.java:265)
at com.intellij.openapi.actionSystem.ex.ActionUtil.performActionDumbAwareWithCallbacks(ActionUtil.java:244)
at com.intellij.ui.popup.ActionPopupStep.performAction(ActionPopupStep.java:253)
at com.intellij.ui.popup.ActionPopupStep.performAction(ActionPopupStep.java:243)
at com.intellij.ui.popup.ActionPopupStep.lambda$onChosen$2(ActionPopupStep.java:229)
at com.intellij.openapi.application.TransactionGuardImpl.performUserActivity(TransactionGuardImpl.java:94)
at com.intellij.ui.popup.AbstractPopup.lambda$dispose$18(AbstractPopup.java:1511)
at com.intellij.util.ui.EdtInvocationManager.invokeLaterIfNeeded(EdtInvocationManager.java:101)
at com.intellij.ide.IdeEventQueue.ifFocusEventsInTheQueue(IdeEventQueue.java:186)
at com.intellij.ide.IdeEventQueue.executeWhenAllFocusEventsLeftTheQueue(IdeEventQueue.java:140)
at com.intellij.openapi.wm.impl.FocusManagerImpl.doWhenFocusSettlesDown(FocusManagerImpl.java:175)
at com.intellij.openapi.wm.impl.IdeFocusManagerImpl.doWhenFocusSettlesDown(IdeFocusManagerImpl.java:36)
at com.intellij.ui.popup.AbstractPopup.dispose(AbstractPopup.java:1508)
at com.intellij.ui.popup.WizardPopup.dispose(WizardPopup.java:164)
at com.intellij.ui.popup.list.ListPopupImpl.dispose(ListPopupImpl.java:326)
at com.intellij.ui.popup.PopupFactoryImpl$ActionGroupPopup.dispose(PopupFactoryImpl.java:271)
at com.intellij.openapi.util.ObjectTree.runWithTrace(ObjectTree.java:136)
at com.intellij.openapi.util.ObjectTree.executeAll(ObjectTree.java:166)
at com.intellij.openapi.util.Disposer.dispose(Disposer.java:169)
at com.intellij.openapi.util.Disposer.dispose(Disposer.java:157)
at com.intellij.ui.popup.WizardPopup.disposeAllParents(WizardPopup.java:268)
at com.intellij.ui.popup.list.ListPopupImpl.handleNextStep(ListPopupImpl.java:433)
at com.intellij.ui.popup.list.ListPopupImpl._handleSelect(ListPopupImpl.java:405)
at com.intellij.ui.popup.list.ListPopupImpl.handleSelect(ListPopupImpl.java:361)
at com.intellij.ui.popup.PopupFactoryImpl$ActionGroupPopup.handleSelect(PopupFactoryImpl.java:288)
at com.intellij.ui.popup.list.ListPopupImpl$MyMouseListener.mouseReleased(ListPopupImpl.java:618)
at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:298)
at java.desktop/java.awt.Component.processMouseEvent(Component.java:6654)
at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3345)
at com.intellij.ui.popup.list.ListPopupImpl$MyList.processMouseEvent(ListPopupImpl.java:695)
at java.desktop/java.awt.Component.processEvent(Component.java:6419)
at java.desktop/java.awt.Container.processEvent(Container.java:2263)
at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5029)
at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321)
at java.desktop/java.awt.Component.dispatchEvent(Component.java:4861)
at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918)
at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547)
at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488)
at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307)
at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2790)
at java.desktop/java.awt.Component.dispatchEvent(Component.java:4861)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:778)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95)
at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:751)
at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:749)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:748)
at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:891)
at com.intellij.ide.IdeEventQueue.dispatchMouseEvent(IdeEventQueue.java:820)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:757)
at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$6(IdeEventQueue.java:447)
at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:818)
at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$7(IdeEventQueue.java:446)
at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:805)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:498)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
Consider showing warning on openapi plugin in IDE if there is no openapi documentation file is not there
Note: sometimes the file can be added in runtime/buildtime or it can have arbitrary path
The generator adds the deprecated Locations plugin when installing OAuth
Added code should be updated to using standard routing or a new Resources plugin.
Sample from docs: https://ktor.io/docs/oauth.html
"Main method, not found in class" when main is a suspend function
What steps will reproduce the problem?
1. Create a new ktor project using default project wizard
2. Add suspend keyword to the main function
3. Create ktor run configuration
4. See error message in run configuration window
Selecting the 'WebSocket-Server' endpoint type in the Endpoints tool window clears all websocket endpoints
To reproduce the issue, open any Ktor project with websocket endpoints. Maybe it's by design by still looks confusing.
"Could not convert string to current locale" error when path contains emoji
To reproduce
- While creating a new project add 😁 at the end of a
Name
field value and select no features. - Run application through the existent run configuration
As a result, I get a compile-time error:
Could not get file details of /home/stexe/projects/ktor_plugin/ktor-sample2😁/build/classes/kotlin/main: could not convert string to current locale
Autoimport for DefaultHeaders conflict with Netty
Bug: when you write DefaultHeaders
statement in the project and call auto-import, there is wrong import priority.
We need to make "io.ktor.server.plugins.defaultheaders" more prioritized then Netty.
Run configuration for newly created projects
For projects that are being created with generator we need to also add run configurations to run the exact main class which is already known.
Support `io.ktor.server.response.ResponseHeaders` to make possible information about headers in responses in OpenAPI generator
Now it is supported old API io.ktor.response.ResponseHeaders
,
in order to support headers in responses in OpenAPI generator support of API io.ktor.server.response.ResponseHeaders
should be added
Ktor wizard generates the yellow code
Ktor project wizard in IDEA generates the following code:
package com.example
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import com.example.plugins.*
import io.ktor.server.application.*
import io.ktor.server.routing.*
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
myApplicationModule()
}.start(wait = true)
}
fun Application.myApplicationModule() {
configureHTTP()
configureTemplating()
configureSerialization()
configureRouting()
}
and it is highlighted as yellow
{width=70%}
"Yellow" code is usually considered a a code with a some kind of problem. But I don't get what is the actually problem with this code? Is generating OpenAPI obligated for Ktor projects?
IntelliJ IDEA 2022.3 EAP (Ultimate Edition)
Build #IU-223.7102, built on October 17, 2022
Licensed to IntelliJ IDEA EAP user: Nicolay Mitropolsky
Expiration date: November 16, 2022
Runtime version: 17.0.4.1+1-b653.1 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 10.14.6
GC: G1 Young Generation, G1 Old Generation
Memory: 4096M
Cores: 16
Metal Rendering is ON
Registry:
ide.cancellation.check.threshold=6000
ide.mac.file.chooser.native=false
ide.unused.symbol.calculation.maxFilesSizeToSearchUsagesIn=20971520
ide.mac.transparentTitleBarAppearance=false
ide.cancellation.check.trace.all=true
scala.erase.compiler.process.jdk.once=false
Non-Bundled Plugins:
org.intellij.terraform.hcl (223.7102)
org.jetbrains.plugins.go-template (223.7102)
com.intellij.kubernetes (223.7102)
com.jetbrains.idea.scalar (223.7102)
org.jetbrains.idea.grammar (2021.1.2)
jetbrains.team.auth (223.7102)
com.intellij.sisyphus (223.7102)
com.jetbrains.idea.safepush (223.7102)
org.intellij.scala (2022.3.378)
Jetbrains TeamCity Plugin (2022.10.115922)
com.jetbrains.intellij.api.watcher (6.62.0)
Kotlin: 223-1.7.20-release-201-IJ7102
An exception is thrown in Endpoints View and OpenAPI generate action for `handle` route
An exception is thrown on an attempt to generate OpenAPI docs for the HttpBin sample: https://github.com/ktorio/ktor-samples/tree/main/httpbin
java.lang.IllegalArgumentException: No enum constant com.intellij.microservices.oas.OasHttpMethod.HANDLE
Intention `Generate open API documentation for current module` duplicates in embedded server entry point
Kdocs with simple tags are invalid in OpenAPI generator.
See https://github.com/ktorio/ktor-samples/blob/11f787613cd95c8aa3db7e786f50f4e8e51bd325/kweet/src/ViewKweet.kt
In this case, the following kdoc:
/**
* This page shows the [Kweet] content and its replies.
* If there is a user logged in, and the kweet is from her/him, it will provide secured links to remove it.
*/
is shown as
content and its replies.
If there is a user logged in, and the kweet is from her/him, it will provide secured links to remove it.
So, everything before the last [***]
is removed from the description.
Add Client Plugins Completion
Copy similar feature we have for the server.
After selecting Generate OpenAPI, IDEA incorrectly shows that documentation already exists
Support wildcard and tailcard parameters in Swagger
Inspection for the case if embeddedServer is used without Application.module function reference
Ex:
embeddedServer(...) {
install(A)
install(B)
}
must be:
embeddedServer(..., module = Application::myModule)
fun Application.myModule() {
install(A)
install(B)
}
Include whole (ktor module) set of configured plugins in generated Ktor routing tests
Example of the current state:
(A*)
embeddedServer(...) {
install(X) { ... }
install(Y) { ... }
configureRouting()
}.start(...)
fun Application.configureRouting() {
routing {
get("/aba") { ... } // <---- generate test for current route
}
}
After the test generation user will see the following test:
@Test
fun testGetFaqArticle() {
withTestApplication({ configureRouting() }) { // <----- X and Y plugins are not installed here
handleRequest(HttpMethod.Get, "/faq/{article?}").apply {
TODO("Please write your test here")
}
}
}
In case when X or Y modifies the application behavior in routing, the test would be incorrect.
In the simplest case, there must be a module already:
(B*)
embeddedServer(...) {
module()
}.start(...)
fun Application.module() {
configureXY()
configureRouting()
}
fun Application.configureXY() {
install(X) { ... }
install(Y) { ... }
}
fun Application.configureRouting() {
routing {
get("/aba") { ... } // <---- generate test for current route
}
}
And the test should call module()
instead of configureRouting
.
Suggestions:
- Add
module
to generator - Add the most upper-level Application extension calls to the test function
- Suggest a refactoring in case of (A*) that transforms the code to (B*)
Migration to the latest version of Ktor doesn't work
When I selected to refactor my project when the pop-up came to go ktor 2.0.3 the imports changes all over BUT the Gradle files were NOT updated, ktor_version = 2.0.2 in gradle.properties and gradle was thus not fetching the new version thus causing a lot of failed imports on the build!
Avoid using Kotlin structuralSearch submodule directly
As it was investigated, Kotlin IDE team has recently split their plugin into multiple submodules PLUS they are migrating kotlin k1 to kotlin k2.
This was done in a way so that and either one of 2 things is true:
- Kotlin plugin it not packed properly => Ktor plugin can not use some of it's modules directly while being built from IntelliJ with JPS because of classloader issue
- Kotlin plugin is packed properly => external plugin developers cannot access it from Gradle plugin.
We need to invent a solution to avoid using it's direct dependencies while still depending on them.
NPE in KtorRoutingVisitor in IJ 2022.3 EAP
- Open Endpoints tool window
- Search for
/api/search/plugins
java.lang.NullPointerException
at io.ktor.ide.KtorRoutingVisitor.processLastArgumentsBodyIfRouteExtensionLambda(KtorUrlResolver.kt:150)
at io.ktor.ide.KtorRoutingVisitor.visitCallExpression(KtorUrlResolver.kt:188)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:176)
at org.jetbrains.uast.UReturnExpression.accept(UReturnExpression.kt:22)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.UMethod.accept(UMethod.kt:45)
at io.ktor.ide.KtorRoutingVisitor.visitCallExpression(KtorUrlResolver.kt:197)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:176)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:180)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:180)
at io.ktor.ide.KtorServerEndpointsProviderKt$collectUrlMappingsInside$processor$1.invoke(KtorServerEndpointsProvider.kt:95)
at io.ktor.ide.KtorServerEndpointsProviderKt$collectUrlMappingsInside$processor$1.invoke(KtorServerEndpointsProvider.kt:94)
at io.ktor.ide.KtorRoutingCallsProcessor.visitCallExpression(KtorServerEndpointsProvider.kt:78)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:176)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:180)
at org.jetbrains.uast.UQualifiedReferenceExpression.accept(UQualifiedReferenceExpression.kt:33)
at org.jetbrains.uast.UReturnExpression.accept(UReturnExpression.kt:22)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:180)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
at org.jetbrains.uast.UMethod.accept(UMethod.kt:45)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.kotlin.AbstractKotlinUClass.accept(AbstractKotlinUClass.kt:99)
at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
at org.jetbrains.uast.UFile.accept(UFile.kt:89)
at io.ktor.ide.KtorServerEndpointsProviderKt.collectUrlMappingsInside(KtorServerEndpointsProvider.kt:97)
at io.ktor.ide.KtorServerEndpointsProviderKt.collectUrlMappingsInside$default(KtorServerEndpointsProvider.kt:91)
at io.ktor.ide.KtorServerEndpointsProvider.getEndpoints(KtorServerEndpointsProvider.kt:46)
at io.ktor.ide.KtorServerEndpointsProvider.getEndpoints(KtorServerEndpointsProvider.kt:21)
at com.intellij.microservices.ui.flat.FlatEndpointGroup.getEndpoints(EndpointsCursor.kt:187)
at com.intellij.microservices.ui.flat.EndpointsCursorKt.getProviderEndpoints(EndpointsCursor.kt:153)
at com.intellij.microservices.ui.flat.EndpointsCursorKt.access$getProviderEndpoints(EndpointsCursor.kt:1)
at com.intellij.microservices.ui.flat.EndpointsCursor$endpointsMap$1.invoke(EndpointsCursor.kt:50)
at com.intellij.microservices.ui.flat.EndpointsCursor$endpointsMap$1.invoke(EndpointsCursor.kt:47)
at com.intellij.microservices.ui.flat.EndpointsCursor.endpointsMap$lambda$2(EndpointsCursor.kt:47)
at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
at java.base/java.util.concurrent.ConcurrentMap.getOrDefault(ConcurrentMap.java:88)
at com.intellij.microservices.ui.flat.EndpointsCursor$request$allSequence$3.invoke(EndpointsCursor.kt:63)
at com.intellij.microservices.ui.flat.EndpointsCursor$request$allSequence$3.invoke(EndpointsCursor.kt:63)
at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:315)
at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)
at kotlin.sequences.FilteringSequence$iterator$1.calcNext(Sequences.kt:169)
at kotlin.sequences.FilteringSequence$iterator$1.hasNext(Sequences.kt:194)
at kotlin.sequences.DropWhileSequence$iterator$1.hasNext(Sequences.kt:557)
at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
at kotlin.sequences.SequenceBuilderIterator.hasNext(SequenceBuilder.kt:115)
at kotlin.sequences.TakeSequence$iterator$1.hasNext(Sequences.kt:421)
at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
at kotlin.sequences.SequencesKt___SequencesKt.toCollection(_Sequences.kt:787)
at kotlin.sequences.SequencesKt___SequencesKt.toMutableList(_Sequences.kt:817)
at kotlin.sequences.SequencesKt___SequencesKt.toList(_Sequences.kt:808)
at com.intellij.microservices.ui.flat.EndpointsView.load$lambda$18(EndpointsView.kt:773)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$OTelMonitor.callWrapped(NonBlockingReadActionImpl.java:746)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$OTelMonitor$MonitoredComputation.call(NonBlockingReadActionImpl.java:778)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:573)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:537)
at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1086)
Fix ActionUpdateThread is missing(Inspection check)
Generate OpenAPI Documentation is reported multiple times.
Having a Ktor code created by wizard:
package com.example
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import com.example.plugins.*
import io.ktor.server.application.*
import io.ktor.server.routing.*
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
myApplicationModule()
}.start(wait = true)
}
fun Application.myApplicationModule() {
configureHTTP()
configureTemplating()
configureSerialization()
configureRouting()
}
I see the inspection reported 5 times on the same element:
{width=70%}
IntelliJ IDEA 2022.3 EAP (Ultimate Edition)
Build #IU-223.7102, built on October 17, 2022
Licensed to IntelliJ IDEA EAP user: Nicolay Mitropolsky
Expiration date: November 16, 2022
Runtime version: 17.0.4.1+1-b653.1 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 10.14.6
GC: G1 Young Generation, G1 Old Generation
Memory: 4096M
Cores: 16
Metal Rendering is ON
Registry:
ide.cancellation.check.threshold=6000
ide.mac.file.chooser.native=false
ide.unused.symbol.calculation.maxFilesSizeToSearchUsagesIn=20971520
ide.mac.transparentTitleBarAppearance=false
ide.cancellation.check.trace.all=true
scala.erase.compiler.process.jdk.once=false
Non-Bundled Plugins:
org.intellij.terraform.hcl (223.7102)
org.jetbrains.plugins.go-template (223.7102)
com.intellij.kubernetes (223.7102)
com.jetbrains.idea.scalar (223.7102)
org.jetbrains.idea.grammar (2021.1.2)
jetbrains.team.auth (223.7102)
com.intellij.sisyphus (223.7102)
com.jetbrains.idea.safepush (223.7102)
org.intellij.scala (2022.3.378)
Jetbrains TeamCity Plugin (2022.10.115922)
com.jetbrains.intellij.api.watcher (6.62.0)
Kotlin: 223-1.7.20-release-201-IJ7102
Support respondFile in OpenAPI documentation
Currently, call.respondFile(...)
is not supported in OpenAPI generator
Support optional path parameters in IDE
get("faq/{article?})
is an optional path parameter that must be reflected in:
- test generation
- endpoints view
- Swagger generation
- Completions
- Other representation
Wizard: Do not include transitive dependency `kotlinx:kotlinx-html-jvm` in build file
Generating a project with the kotlinx HTML DSL for templating creates the following entries in the build file:
implementation("org.jetbrains.kotlinx:kotlinx-html-jvm:0.7.3")
implementation("io.ktor:ktor-html-builder:$ktor_version")
ktor-html-builder
already includes the dependency on kotlinx-html-jvm
, so I it does not seem necessary to duplicate this declaration in the build file:
$ ./gradlew -q dependencies
...
+--- io.ktor:ktor-html-builder:1.5.3
| +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.32 (*)
| +--- org.slf4j:slf4j-api:1.7.30
| +--- org.jetbrains.kotlinx:kotlinx-html-jvm:0.7.3 (*)
| \--- io.ktor:ktor-server-core-kotlinMultiplatform:1.5.3 (*)
Server
Intergate Swagger UI Hosting as Ktor Feature
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/453
It would be really nice if Ktor could support hosting a Swagger UI that is generated from your routes
configuration.
For example, the following could be used to generate a Swagger UI.
data class PetModel(val id: Int?, val name: String)
data class PetsModel(val pets: MutableList<PetModel>)
val data = PetsModel(mutableListOf(PetModel(1, "max"), PetModel(2, "moritz")))
fun newId() = ((data.pets.map { it.id ?: 0 }.max()) ?: 0) + 1
@Group("pet operations")
@Location("/pets/{id}")
class pet(val id: Int)
@Group("pet operations")
@Location("/pets")
class pets
@Group("debug")
@Location("/request/info")
class requestInfo
@Group("debug")
@Location("/request/withHeader")
class withHeader
class Header(val optionalHeader: String?, val mandatoryHeader: Int)
@Group("debug")
@Location("/request/withQueryParameter")
class withQueryParameter
class QueryParameter(val optionalParameter: String?, val mandatoryParameter: Int)
fun main(args: Array<String>) {
val server = embeddedServer(Netty, getInteger("server.port", 8080)) {
install(DefaultHeaders)
install(Compression)
install(CallLogging)
install(ContentNegotiation) {
gson {
setPrettyPrinting()
}
}
install(Locations)
install(SwaggerSupport) {
forwardRoot = true
swagger.info = Information(
version = "0.1",
title = "sample api implemented in ktor",
description = "This is a sample which combines [ktor](https://github.com/Kotlin/ktor) with [swaggerUi](https://swagger.io/). You find the sources on [github](https://github.com/nielsfalk/ktor-swagger)",
contact = Contact(
name = "Niels Falk",
url = "https://nielsfalk.de"
)
)
}
routing {
get<pets>("all".responds(ok<PetsModel>())) {
call.respond(data)
}
post<pets, PetModel>("create".responds(ok<PetModel>())) { _, entity ->
// http201 would be better but there is no way to do this see org.jetbrains.ktor.gson.GsonSupport.renderJsonContent
call.respond(entity.copy(id = newId()).apply {
data.pets.add(this)
})
}
get<pet>("find".responds(ok<PetModel>(), notFound())) { params ->
data.pets.find { it.id == params.id }
?.let {
call.respond(it)
}
}
put<pet, PetModel>("update".responds(ok<PetModel>(), notFound())) { params, entity ->
if (data.pets.removeIf { it.id == params.id && it.id == entity.id }) {
data.pets.add(entity)
call.respond(entity)
}
}
delete<pet>("delete".responds(ok<Unit>(), notFound())) { params ->
if (data.pets.removeIf { it.id == params.id }) {
call.respond(Unit)
}
}
get<requestInfo>(
responds(ok<Unit>()),
respondRequestDetails()
)
get<withQueryParameter>(
responds(ok<Unit>())
.parameter<QueryParameter>(),
respondRequestDetails()
)
get<withHeader>(
responds(ok<Unit>())
.header<Header>(),
respondRequestDetails()
)
}
}
server.start(wait = true)
}
fun respondRequestDetails(): suspend PipelineContext<Unit, ApplicationCall>.(Any) -> Unit {
return {
call.respond(
mapOf(
"parameter" to call.parameters,
"header" to call.request.headers
).format()
)
}
}
private fun Map<String, StringValues>.format() =
mapValues {
it.value.toMap()
.flatMap { (key, value) -> value.map { key to it } }
.map { (key, value) -> "$key: $value" }
.joinToString(separator = ",\n")
}
.map { (key, value) -> "$key:\n$value" }
.joinToString(separator = "\n\n")
This is the swagger UI generated from the above.
I spent today overhauling @nielsfalk's project ktor-swagger to use the newest version of Ktor and also use Gradle to build the application PR here.
I think this project has quite a bit of potential and could satisfy a need in the community by allowing for a fast way to create documentation for API's written using Ktor.
If the Ktor team would like to adopt this project as a feature, I'm happy to try to make the port from the external project it is today into this repository.
If the interest does not exist to adopt a new feature, I totally understand. The concern that I have with publishing this myself (or with @nielsfalk assistance) is the issue of incompatible breaking changes in Ktor (as Ktor is pre-1.0).
I open the floor to the developers of this project. I'd love to see this integrated as a fully supported feature, but I understand if this is outside the scope of this project.
Add a method that returns a list of child routes recursively
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1252
https://github.com/ktorio/ktor/issues/788Subsystem
Server
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Things like spring actuator are able to list all the endpoints in the current system. Something like this for ktor would be very helpful.
Motivation to include to ktor
A very common usage so makes sense to build directly into ktor.
Rate-Limit Support on Server
Create a plugin that will limit the number of requests in time for the server and for clients with different throttling strategies.
It would be great to have logging for the RPS metric.
No way getting client's source address from IP packet
From the route handler context I want to get a client's IP address but unfortunately, call.request.origin.remoteHost
returns symbolic name in some cases, e.g. localhost
.
A relevant question on the StackOverflow https://stackoverflow.com/questions/67054276/how-to-get-client-ip-with-ktor
Support serving OpenAPI from resources
resources/openapi/documentation.yaml
should be the default path for OpenAPI in Ktor
Restoring thread context elements when directly resuming to parent is broken
There are some known and fixed issues about restoring of ThreadLocal value in coroutines.
It was discussed here - https://github.com/Kotlin/kotlinx.coroutines/issues/985 and (as far as I understand) fixed here - https://github.com/Kotlin/kotlinx.coroutines/commit/5dc55a641ada3a45f0aa3a46a30bd4483c48cebf with a big test - https://github.com/Kotlin/kotlinx.coroutines/blob/5dc55a641ada3a45f0aa3a46a30bd4483c48cebf/kotlinx-coroutines-core/jvm/test/ThreadContextElementRestoreTest.kt
But in my KTOR-based project, I have started to receive issues with non-cleared ThreadLocal values. After some work, I was able to create small reproducible projects (in attachment).
It seems like async IO (httpClient().request<String>
) somehow affects restoring of thread-local values. I have no idea how to reproduce it without KTOR.
Basically, it fails on next code part:
suspend fun <T> withRequestId(requestId: Int, code: suspend CoroutineScope.() -> T): T {
if (requestIdThreadLocal.get() != null) {
throw IllegalStateException("thread local is set somehow?")
}
val result =
withContext(requestIdThreadLocal.asContextElement(requestId)) {
code()
}
if (requestIdThreadLocal.get() != null) {
// https://github.com/Kotlin/kotlinx.coroutines/issues/985#issuecomment-534929877
throw IllegalStateException("Oh, it should be fixed in kotlin already! $requestId ${requestIdThreadLocal.get()}")
}
return result
}
at the same time ThreadContextElementRestoreTest
proves next in different configurations:
withContext(tl.asContextElement("OK")) {
block()
assertEquals("OK", tl.get())
}
assertEquals(null, tl.get())
Tested on openjdk 11.0.7 2020-04-14
UPD: https://github.com/michail-nikolaev/kotlin-coroutines-thread-local
Inconsistency among server engines when determining port/host of an incoming request
The three server engines Netty, CIO and Jetty use very different behaviour to determine the port and host of an incoming request.
CIO: always tries to get the hostname and port of the local IP endpoint first and only if the fails tries to parse the host header.
Netty HTTP1: parses the host header first for the hostname, falls back to local IP endpoint if that fails. For the port it always uses the one from the local IP endpoint and never consults the host header.
Netty HTTP2: defaults to authority header, falls back to local IP endpoint when authority fails for the port.
Jetty: uses the servletRequest which seems to prefer the host header for port and host.
I think all engines should behave consistently. Ideally all engines prefer the host header from the request and only fall back to the local IP endpoint if that fails. Otherwise running ktor behind a TCP load balancer is really hard because the local IP endpoint might not match the one of the incoming request.
ApplicationCall.respondRedirect should have overload for Url
There this one:
public suspend fun ApplicationCall.respondRedirect(url: String, permanent: Boolean = false)
and there should also be an overload that takes a Url
, as that's the primary type to use when dealing with URLs.
public suspend fun ApplicationCall.respondRedirect(url: Url, permanent: Boolean = false)
Add a way to get a client's port
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1754
I can get client host by call.request.local.remoteHost
but how to get client port?
Netty: ApplicationStarted event is fired before the server starts accepting connections
We use ktor in our application and have written a set of integration tests that utilize the ktor client to connect to our server. These recently started failing for me on my mac and I noticed that the ApplicationStarted monitor was notified but the server is rejecting the connection. I have created a minimal example in the following repo [1]. I have found I need to increase the client delay ms field to around 5500 in order for this test to pass on my macbook. On my PC this is fine with 1ms of delay.
System: macOS 11.6.5, 2.4GHz 8-core Intel Core i9, 64 GB RAM, kotlin version 1.6.21, ktor version 2.0.1
Steps to reproduce:
1. Clone repo
2. mvn test
Expected:
Both assert pass
Actual:
The client receives an exception,
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:50)
at io.ktor.network.sockets.SocketImpl$connect$1.invokeSuspend(SocketImpl.kt)
(Coroutine boundary)
at io.ktor.client.engine.cio.Endpoint.execute(Endpoint.kt:76)
at io.ktor.client.engine.cio.CIOEngine.execute(CIOEngine.kt:80)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:97)
at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:98)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:135)
at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:61)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:149)
at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:101)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:125)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:35)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:107)
at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:46)
at KtorClientTest$getData$2$1.invokeSuspend(KtorTest.kt:96)
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:50)
at io.ktor.network.sockets.SocketImpl$connect$1.invokeSuspend(SocketImpl.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Application data in OAuth State parameter
As I understand it, the OAuth state parameter is used as a nonce for the authentication process to prevent CSRF attacks. However, with the current interface usable to override the default implementation of NonceManager, there is no request context available, and the state is not included in the OAuth response principal. This prevents any attempt to include application state in the state parameter. This is necessary for many use-cases, the most common being the ability to specify an extra redirect_uri for returning the user to whatever page they were attempting to access prior to the login flow being initiated.
Not calling call.respond() at server results in 404 for the client
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/737
Calling a service from the client results in a 404-error if the server does not respond with a
call.respond()
404 Not Found is a bit missleading here.
ktorVersion = "1.0.0-beta-3"
Discussion at Slack: https://kotlinlang.slack.com/archives/C0A974TJ9/p1542617591326600
Allow nested authentications to be combined using AND
Usecase:
I'm developing a website where I have a session authentication block that checks whether the user has a session and validates it. I also have an oauth authentication block that allows for authentication using a different platform (like Google, eBay, Xero, etc).
authentication {
session<UserSession>("session-auth") { /* block */ }
oauth("ebay-oauth") { /* block */ }
}
install(Sessions) {
cookie<UserSession>("user_session")
}
The idea is that I want to combine these two authentication blocks, so that it should first check if the user has a session on the website (if they are logged in), and if so then they should be able to authenticate using oauth. This is so that I can associate the oauth tokens given to a user id that's stored on the session.
routing {
authenticate("session-auth") {
authenticate("ebay-oauth") {
get("/ebay/login") {
// redirect to oauth authorize url
}
get("/ebay/login/callback") {
val session = call.principal<UserSession>()
val principal = call.principal<OAuthAccessTokenResponse.OAuth2>()
val userId = session.userId
// save tokens in database
}
}
}
}
The behaviour right now is that when the user has a session, it will skip the ebay oauth authentication block and go straight into the route. If there is no session, it will lead the user to the oauth authentication url. This essentially means that combined authentication blocks act as an 'OR' operator. I would like an option to combine authentication blocks using an AND operator, or maybe a whole new function like authenticateAll
which should go through each authentication block 1 by 1 and make sure they all work.
If this is not possible, could you suggest a solution for the problem I have?
Thanks!
The swaggerUI plugin should be placed in the io.ktor.server.plugins.swagger package
Currently, it's placed in io.ktor.server.swagger
.
The default path to an OpenAPI specification doesn't work for the 'openAPI' plugin
On an attempt to use OpenAPI docs placed in the default location, the following error is thrown:
openapi/documentation.yaml (No such file or directory)
java.io.FileNotFoundException: openapi/documentation.yaml (No such file or directory)
Chaning the path as follows helps: src/main/resources/openapi/documentation.yaml
.
P.S.: At the same time, swaggerUI
works fine with the default openapi/documentation.yaml
path.
JWT: JWTPayloadHolder.getListClaim() throws NPE when specified claim is absent
The jwt-auth plugin will NPE if a list claim is null.
Claim::asList
(Java method) will return null
if the claim isn't an array, but JWTPayloadHolder::getListClaim
returns the result directly, even though its signature doesn't allow a nullable return type.
SessionTransportTransformerMessageAuthentication: Comparison of digests fails when there is a space in a value
In transformRead
, spaces in the session data in transportValue
(read from a cookie) are not encoded and are given as plain spaces.
In transformWrite
, however, spaces in transportValue
are encoded using "+".
The result is that the hashes never match when there are spaces in the session data.
Add Server BearerAuthenticationProvider
While Ktor provides mechanisms for authorizing JWTs and OAuth tokens, sometimes the developer needs to authorize bearer tokens with a custom lookup. I propose a new BearerAuthenticationProvider
that extracts the bearer token from the Authorization header, and delegates the Principal lookup to the provided AuthenticationFunction<String>
.
I already have a working prototype I can contribute.
Merged config: "Property *.size not found" error when calling `configList` method on an array property
Given the following HOCON config:
some {
arr = [1, 2, 3]
}
To reproduce run the following code on a server that uses the above config:
environment.config.configList("some.arr")
As a result, an unexpected ApplicationConfigurationException
is thrown:
Exception in thread "main" io.ktor.server.config.ApplicationConfigurationException: Property some.arr.size not found.
at io.ktor.server.config.MapApplicationConfig.configList(MapApplicationConfig.kt:57)
at io.ktor.server.config.MergedApplicationConfig.configList(MergedApplicationConfig.kt:37)
at EngineMainKt.module(EngineMain.kt:6)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at kotlin.reflect.jvm.internal.calls.CallerImpl$Method.callMethod(CallerImpl.kt:97)
at kotlin.reflect.jvm.internal.calls.CallerImpl$Method$Static.call(CallerImpl.kt:106)
at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:108)
at kotlin.reflect.jvm.internal.KCallableImpl.callDefaultMethod$kotlin_reflection(KCallableImpl.kt:159)
at kotlin.reflect.jvm.internal.KCallableImpl.callBy(KCallableImpl.kt:112)
at io.ktor.server.engine.internal.CallableUtilsKt.callFunctionWithInjection(CallableUtils.kt:119)
at io.ktor.server.engine.internal.CallableUtilsKt.executeModuleFunction(CallableUtils.kt:36)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:335)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:334)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:359)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.launchModuleByName(ApplicationEngineEnvironmentReloading.kt:334)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$launchModuleByName(ApplicationEngineEnvironmentReloading.kt:32)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:315)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:313)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:341)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:313)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:150)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:280)
at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:211)
at io.ktor.server.netty.EngineMain.main(EngineMain.kt:26)
at EngineMainKt.main(EngineMain.kt:3)
Add the ability to access the route inside a route-scoped plugin
Right now there is no way to access the route into which a route-scoped plugin is installed.
https://kotlinlang.slack.com/archives/C0A974TJ9/p1667243423047399
https://kotlinlang.slack.com/archives/C0A974TJ9/p1651935166437179
StatusPages can't handle errors in HTML template
Given the following code
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
class MyTemplate : Template<HTML> {
override fun HTML.apply() {
head {
title("Minimum Working Example")
}
body {
throw Exception("Error")
}
}
}
fun Application.module() {
install(Sessions) {
cookie<String>("SESSION", SessionStorageMemory()) {
cookie.extensions["SameSite"] = "lax"
}
}
install(StatusPages) {
exception<Throwable> { call, cause ->
call.respond(HttpStatusCode.InternalServerError, cause.message.toString())
}
}
routing {
get("/") {
call.respondHtmlTemplate(MyTemplate()) {}
}
}
}
There seems to be no way to check for already sent headers before trying to set cookies on the response sent which results in the error
Headers can no longer be set because response was already completed
It appears the header for status code was already sent by the point the intercept happens.
What is the correct way to do this, or alternatively is there a way to implement a check if the headers were already sent and then abort the cookie setting?
2022-11-04 14:47:43.202 [main] INFO ktor.application - Autoreload is disabled because the development mode is off.
2022-11-04 14:47:43.609 [main] INFO ktor.application - Application started in 0.467 seconds.
2022-11-04 14:47:43.950 [DefaultDispatcher-worker-2] INFO ktor.application - Responding at http://127.0.0.1:8080
2022-11-04 14:47:45.297 [eventLoopGroupProxy-4-1] DEBUG ktor.application - Failed to lookup session: Session cdad17cf51193d184020bff9a4491f8f not found. The session id is wrong or outdated.
2022-11-04 14:47:45.346 [eventLoopGroupProxy-4-1] ERROR ktor.application - 500 Internal Server Error: GET - /
java.lang.UnsupportedOperationException: Headers can no longer be set because response was already completed
at io.ktor.server.netty.http1.NettyHttp1ApplicationResponse$headers$1.engineAppendHeader(NettyHttp1ApplicationResponse.kt:42)
at io.ktor.server.response.ResponseHeaders.append(ResponseHeaders.kt:57)
at io.ktor.server.response.ResponseHeaders.append$default(ResponseHeaders.kt:48)
at io.ktor.server.response.ResponseCookies.append(ResponseCookies.kt:33)
at io.ktor.server.sessions.SessionTransportCookie.clear(SessionTransportCookie.kt:57)
at io.ktor.server.sessions.SessionDataKt.sendSessionData(SessionData.kt:151)
at io.ktor.server.sessions.SessionsKt$Sessions$2$2.invokeSuspend(Sessions.kt:59)
at io.ktor.server.sessions.SessionsKt$Sessions$2$2.invoke(Sessions.kt)
at io.ktor.server.sessions.SessionsKt$Sessions$2$2.invoke(Sessions.kt)
at io.ktor.server.sessions.BeforeSend$install$1.invokeSuspend(Sessions.kt:14)
at io.ktor.server.sessions.BeforeSend$install$1.invoke(Sessions.kt)
at io.ktor.server.sessions.BeforeSend$install$1.invoke(Sessions.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:101)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at ApplicationKt$module$2$1.invokeSuspend(Application.kt:58)
at ApplicationKt$module$2$1.invoke(Application.kt)
at ApplicationKt$module$2$1.invoke(Application.kt)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2$1.invokeSuspend(StatusPages.kt:68)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2$1.invoke(StatusPages.kt)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2$1.invoke(StatusPages.kt)
at io.ktor.server.logging.EmptyMDCProvider.withMDCBlock(Logging.kt:27)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2.invokeSuspend(StatusPages.kt:67)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2.invoke(StatusPages.kt)
at io.ktor.server.plugins.statuspages.StatusPagesKt$StatusPages$2$2.invoke(StatusPages.kt)
at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:46)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:33)
at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:288)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
CallLogging and CallId: exceptions thrown in WriteChannelContent.writeTo are swallowed
Tested on Ktor 2.1.0, 2.1.1, 2.1.2.
Plugin CallId from io.ktor:ktor-server-call-id-jvm:2.1.2
.
It seems, that using a project created by the IntelliJ IDEA wizard for Ktor selecting call logging and call ID plugins, the generated code looks something like (code compacted, some functions renamed etc. in this issue textfield. Hope the syntaxt is indeed valid)
fun Application.configureMonitoring() {
install(CallLogging) {
level = Level.INFO
filter { call -> call.request.path().startsWith("/") }
callIdMdc("call-id")
}
install(CallId) {
header(HttpHeaders.XRequestId)
verify { callId: String ->
callId.isNotEmpty()
}
}
}
fun main(args: Array<String>): Unit =
io.ktor.server.netty.EngineMain.main(args)
@Suppress("unused")
fun Application.module() {
//configureHttp()
configureMonitoring()
//configureTemplating()
//configureRouting()
configurePage()
}
private fun Application.configurePage() {
routing {
route("/page") {
get {
call.respondHtml {
//defaultHead()
body {
pageForm()
//initScripts()
}
}
}
}
}
}
private fun BODY.pageForm() = form {
// ...
select {
// omitted, not important
name = "page-select"
}
}
Would throw
2022-10-07 10:02:13.872 [eventLoopGroupProxy-4-1] ERROR
Application - 200 OK: GET - /page
java.lang.IllegalStateException: You can't change tag attribute because it was already passed to the downstream
at kotlinx.html.consumers.DelayedConsumer.onTagAttributeChange(delayed-consumer.kt:16)
at kotlinx.html.impl.DelegatingMap.put(delegating-map.kt:27)
at kotlinx.html.impl.DelegatingMap.put(delegating-map.kt:5)
at kotlinx.html.attributes.Attribute.set(attributes.kt:19)
at kotlinx.html.SELECT.setName(gen-tags-s.kt:120)
But with the CallId
plugin install
ed, all stack traces are consumed and only the call is logged and browser waits until timeout and then reports that the page didn't send any content
Commenting out the CallId
plugin, the stack traces come back and everything works.
Temp files generated by multipart upload are not cleared in case of exception or cancellation
STR:
- Start upload big file with multipart upload
- Cancel it in the middle
AR:
tmp file generated byCIOMultipartDataBase.kt
not deleted
ER:
tmp file generated byCIOMultipartDataBase.kt
is deleted
Digest auth: Support returning any objects which implement Principal interface
Hi,
when using digest authentication, you cannot set a validate method to map the credentials to a principal. You can only use the default principal UserIdPrincipal.
In contrast when using basic or jwt authorization, you can map the credentials to a custom principal. Is there any reason that the validate method is missing? Maybe add it so you can use a custom principal with digest authorization?
Thanks!
Server Session - Switch to Kotlinx serialization
Currently, the session feature uses a lot of reflection and custom hardcoded JVM serializer. Switching to the KotlinxSerialization would remove all the reflection, allow easy custom serializer with different classes (eg kotlinx.Instant) and different formats.
Very edge design
fun <T> KSerializer<T>.toSessionSerializer(format: StringFormat) = object : SessionSerializer<T> {
override fun deserialize(text: String): T = format.decodeFromString(this@toSessionSerializer, text)
override fun serialize(session: T): String = format.encodeToString(this@toSessionSerializer, session)
}
inline fun <reified T : Any> Sessions.Configuration.cookie(
name: String,
serializer: KSerializer<T>,
format: StringFormat,
block: CookieConfiguration.() -> Unit
) {
val builder = CookieConfiguration().apply(block)
register(
provider = SessionProvider(
name = name, type = T::class,
tracker = SessionTrackerByValue(T::class, serializer = serializer.toSessionSerializer(format)),
transport = SessionTransportCookie(name, configuration = builder, transformers = emptyList())
)
)
}
Way to block use of TLS 1.0/1.1 when using Ktor/Netty
As per following discussion it doesn't look like there's way to block use of TLS 1.0/1.1 when using Ktor/Netty
https://kotlinlang.slack.com/archives/C0A974TJ9/p1656335256667499
FWIW I modified/built local version of Ktor that included protocols("TLSv1.3", "TLSv1.2")
in NettyChannelInitializer
and that seemed to work.
Unable to access userPrincipal of servletRequest in ktor-server-servlet
In io.ktor.server.servlet.JAASBridge.kt
, an extension function is defined that extracts the servletRequest.userPrincipal
if the request is of the type ServletApplicationRequest
.
public val ApplicationRequest.javaSecurityPrincipal: Principal?
get() = when (this) {
is ServletApplicationRequest -> servletRequest.userPrincipal
else -> null
}
To extract the Servlet prinicipal from Ktor route handlers, you can call this with call.request.userPrincipal
.
No matter how you call javaSecurityPrincipal
, it seems that you always get null
in return.
As a part of the routing stack, the request object ends up being wrapped as a RoutingApplicationRequest
(see io/ktor/server/routing/RoutingApplicationCall.kt
). This class uses delegation:
public class RoutingApplicationRequest(
override val call: RoutingApplicationCall,
override val pipeline: ApplicationReceivePipeline,
request: ApplicationRequest
) : ApplicationRequest by request
This causes the check is ServletApplicationRequest
in ApplicationRequest.javaSecurityPrincipal
to never succeed, because is
does not seem to have any knowledge of delegation, and only checks the class hierarchy.
When unable to get JWKS, JWTAuth swallows the underlying exception and only logs the last message
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1053
When unable to retrieve the JWKS, JWTAuth swallows the underlying exception and only logs the message.
This means we only get to see:
TRACE io.ktor.auth.jwt - Failed to get JWK: Failed to get key with kid 1
When stack trace is:
com.auth0.jwk.SigningKeyNotFoundException: Failed to get key with kid 1
Caused by: com.auth0.jwk.SigningKeyNotFoundException: Cannot obtain jwks from url https://localhost:2222/jwks
Caused by: java.net.ConnectException: Connection refused (Connection refused)
Simple example to reproduce:
val TestJWT = "eyJraWQiOiIxIiwiYWxnIjoiUlMyNTYifQ.eyJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwNjAvaXNzby9vYXV0aDIiLCJleHAiOjE3MTE4NzQ3MDAsImp0aSI6IjZvWkZWa3lGX2RSd1dpcXNhQkJkWkEiLCJpYXQiOjE1NTQxOTQ3MDAsInN1YiI6Im15dXNlciIsImF1ZCI6Ik9JREMiLCJhY3IiOiJMZXZlbDQiLCJhenAiOiJPSURDIn0.NuWfe1BZK3i-VVC1l7EIvJydd9m3Pcr2_0AanhbS3YEXSq_NWKhqtFd4qM_KUhLURTTwhNhAb43Zr2HzxGFUhnYnU4uCi95fLcw3Cq8mTM3o4I0r-pgpPTkfiheUUtOA4d43cwWpyEaBdypwO_F-VLA4zBw1oTRE_M0_G-16Q6yezpjTVBvOI7nsEWLHUZ-i10hE3V53cx2-Qm5OUOtEFF-UqqFhgBU6VSRYS5J3puWQFGlLr5hGSAW3Nll1DkJbiNaHB4y7EPnSlCPcNdZ98PXckylsiJ6nhRJXg4mke-C2WWckJ5H4dgsjeoUmXDuLekO1IrvwT1JLGJYiPwlQJw"
fun main(args: Array<String>): Unit {
Thread {
io.ktor.server.netty.EngineMain.main(args)
}.run()
Thread {
println("Testing in 3 secs...")
Thread.sleep(3000);
println("Testing...")
GlobalScope.launch {
val resp = HttpClient(Apache).use { client ->
client.get<String>(URL("http://localhost:8080/")) {
this.header("Authorization", "Bearer " + TestJWT);
}
}
println("RESP: ${resp}")
}
}.run()
}
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
install(Authentication) {
jwt {
verifier(JwkProviderBuilder(URL("https://localhost:2222/jwks")).build(), "http://localhost:8060/isso/oauth2");
validate { credentials ->
if (credentials.payload.subject == "myuser") {
JWTPrincipal(credentials.payload)
} else {
log.info("${credentials.payload.subject} is not authorized to use this app, denying access")
null
}
}
}
}
routing {
authenticate {
get("/") {
call.respondText("HELLO WORLD!", contentType = ContentType.Text.Plain)
}
}
}
}
CIO Server generates wrong URL for OAuth URL provider using Locations
Following the OAuth docs, I noticed that the redirect URI would always be wrong when using CIO server with locations.
When doing url(login(it.name))
in the urlProvider
for the OAuth, the CallbackUrl
will become http://localhost:8080/login/type
with Netty, but http://0:8080/login/type
with CIO.
Code example:
fun main() {
embeddedServer(CIO, module = Application::module, port = 8080).start(true)
}
fun Application.module() {
oauth("oauth") {
this.client = client
providerLookup = {
providers[application.locations.resolve<Login>(Login::class, this).provider]
}
urlProvider = {
// This is where the issue happens
url(Login(it.name))
}
}
}
@Location("/login/{provider}")
data class Login(val provider: String)
val providers = listOf(
OAuthServerSettings.OAuth2ServerSettings(
name = "github",
authorizeUrl = "https://github.com/login/oauth/authorize",
accessTokenUrl = "https://github.com/login/oauth/access_token",
clientId = "***",
clientSecret = "***"
)
).associateBy {it.name}
The code is basically a straight copy from the docs.
Test Infrastructure
testApplication: application initialization block isn't eagerly called
When migrating from the old withTestApplication
to the new testApplication
, it broke my tests, because the blocks for configuration of routing
and application
are not executed, or only executed after the test implementation finished. When looking into the source code, it might have to do with the engine
only being called/initialized after the test block. Please look at the tests below comapring the old and new approach:
import io.ktor.server.testing.*
import io.ktor.test.dispatcher.*
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
internal class CompareTestApplicationTest {
@Test
fun `New testApplication initialization is called (failing)`() = testSuspend {
var externalServicesCalled = false
var routingCalled = false
var applicationCalled = false
testApplication {
externalServices {
externalServicesCalled = true
println("Always called, because this executes instantly")
}
routing {
routingCalled = true
println("Only called AFTER testApplication { } block finished")
}
application {
applicationCalled = true
println("Only called AFTER testApplication { } block finished")
}
assertEquals(true, externalServicesCalled) // success
assertEquals(true, routingCalled) // failure
assertEquals(true, applicationCalled) // failure
println("testApplication { } block finished")
}
}
@Test
fun `Deprecated withTextApplication initialization is called (success)`() = testSuspend {
var applicationCalled = false
withTestApplication({
applicationCalled = true
println("Properly called before the actual tests below")
}) {
assertEquals(true, applicationCalled)
println("withTestApplication { } block finished")
}
}
}
Using Ktor 2.1.0, Kotlin 1.7.10, junit-jupiter 5.9.0
It also seems that if I run multiple tests in parallel, the behavior differs compared to running tests individually. Is there some sort of caching/reusing going on, that might break tests? It should be possible to define different setups for the testApplication
block for different tests, right?
testApplication: test server lifecycle management
Following the instructions in Ktor Server > Testing (step 7) my testing setup code looks like this:
private fun inMemoryTestBroken(testBlock: suspend ApplicationTestBuilder.(HttpClient) -> Unit) {
testApplication {
environment {
config = MapApplicationConfig(
"requests.heartbeatTtl" to "PT5S",
"requests.timeout" to "PT5S",
"requests.pollSleep" to "PT0.1S",
)
}
application {
// fields in the test class
attributes.put(Connector, testInjector)
attributes.put(AppClock, coroutineClock)
// creates a service, but does not start the event dispatching thread
// until the ApplicationStarted event has been broadcasted
installService()
configureRouting()
}
val client = createClient {
install(ContentNegotiation) { json() }
}
// the actual test is executed in this block as in the example
testBlock(client)
}
}
As the testApplication
block is executed during the application builder config, the application is not ready at the time we run the testBlock
. This means that some of the service resources have not been allocated and our tests are failing.
As a second attempt, I tried to use the test client directly:
private fun inMemoryTestBroken(testBlock: suspend ApplicationTestBuilder.(HttpClient) -> Unit) {
val testApplication = TestApplication {
environment { // same as in the first snippet, reformatted for more compact view
config = MapApplicationConfig("requests.heartbeatTtl" to "PT5S", "requests.timeout" to "PT5S", "requests.pollSleep" to "PT0.1S", )
}
application { // same as in the first snippet, reformatted for more compact view
attributes.apply { put(Connector, testInjector); put(AppClock, coroutineClock) }; installService() ; configureRouting()
}
}
try {
// PROBLEM: engine is internal and fails to compile
testApplication.engine.start()
// PROBLEM: testApplication.builder is private and fails to compile
// I also tried to create a fresh one, but it fails to link with the test application
val builder = testApplication.builder
val client = builder.createClient {
install(ContentNegotiation) { json() }
}
runBlocking {
builder.testBlock(client)
}
} finally {
testApplication.stop()
}
}
As you can see in the comments, critical parts of the API are inaccessible.
It feels dirty to start threads during module installation time, but right now that is the only thing that works. Also, I am not convinced that starting my test from the init block does not have any other side effects.
Please let me known if there is any other way.
Other
Add Debug Logging to ContentNegotiation
Add Debug Logging to WebSockets Plugin
Add Debug Logging for Ktor Plugins and Routing
Exceptions during request receive are hard to debug. To simplify logging we can introduce debug logging with reason what happens with the request.
Update Versions of Dependencies
Update slf4j to 2.0.4
Update yamlkt to 0.12.0
Update xmlutil to 0.84.3
Update thymeleaf to 3.1.1.RELEASE
Update webjars-jquery to 3.6.1
Update webjars-locator to 0.52
Update pebble to 3.1.6
Update jwks-rsa to 0.21.2
Update jwt to 3.19.3
Update Mockk to 1.13.3
Update Micrometer to 1.10.2
Update Dropwizard to 4.2.13
Update Logback to 1.3.5
Update Jackson to 2.14.1
Update Gson to 2.10.0
Update OkHttp to 4.10.0
Update Tomcat to 9.0.70
Update Jetty to 9.4.49.v20220914
Update Netty to 4.1.85.Final
Update kotlinx.serialization to 1.4.1
Update atomicfu to 0.18.4
Update kotlinx.html to 0.8.0
Update Kotlin to 1.7.22
Update Gradle to 7.5.1
ByteChannel exception: Got EOF but at least 1 bytes were expected
There's a problem with jvm ByteChannel which is reproduciple with ktor 2.1.3
and 2.2.0-eap-553
and doesn't reproducible with 2.0.2-eap-393
.
Code in the snippet throws Got EOF but at least 1 bytes were expected
exception at a random attempt.
import io.ktor.utils.io.*
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
fun main(args: Array<String>) {
runBlocking {
(1..20000).forEach { num ->
val channel = ByteChannel(true)
val writer = launch(Dispatchers.IO) {
channel.writeFully("1\n".toByteArray())
channel.close()
}
val reader = async(Dispatchers.IO) {
val lines = mutableListOf<String>()
while (true) {
val line = channel.readUTF8Line(5000) ?: break
lines.add(line)
}
lines
}
val readerResult = reader.await()
writer.join()
println("[$num] READER RESULT: $readerResult")
}
}
}
Full stacktrace:
Exception in thread "main" java.io.EOFException: Got EOF but at least 1 bytes were expected
at io.ktor.utils.io.ByteBufferChannel.readBlockSuspend(ByteBufferChannel.kt:1707)
at io.ktor.utils.io.ByteBufferChannel.read$suspendImpl(ByteBufferChannel.kt:1661)
at io.ktor.utils.io.ByteBufferChannel.read(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteBufferChannel.readUTF8LineToUtf8Suspend(ByteBufferChannel.kt:1952)
at io.ktor.utils.io.ByteBufferChannel.readUTF8LineToAscii(ByteBufferChannel.kt:1938)
at io.ktor.utils.io.ByteBufferChannel.readUTF8LineTo$suspendImpl(ByteBufferChannel.kt:2006)
at io.ktor.utils.io.ByteBufferChannel.readUTF8LineTo(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteBufferChannel.readUTF8Line$suspendImpl(ByteBufferChannel.kt:2010)
at io.ktor.utils.io.ByteBufferChannel.readUTF8Line(ByteBufferChannel.kt)
at MainKt$main$1$1$reader$1.invokeSuspend(Main.kt:19)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Also I've attached full gradle project, so you can just unarchive it and run code via ./graldew run
in the project root.
Support Default Value for missing Env Variables in YAML
The syntax can be similar to $NAME:default
value or ${NAME:default_value}
Out of the box ContentConverter for Protobuf
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/517
Currently only JSON converters supported out of the box (Gson, Jackson)
It would be nice to have Protobuf converter to be able to use (kotlin + retrofit + protobuf) on android client side and (ktor + protobuf) on server side.
It is quite common use case to migrate from Json to Protobuf in mobile projects in case of large amount of small objects passing to requests/responses. The reasons are simple - improved parsing performance, lower CPU usage than gson/jackson, fixed protocol, api versioning etc.
On Android client side Retrofit supported Proto converter more than a year ago - https://github.com/square/retrofit/blob/master/retrofit-converters/protobuf/src/main/java/retrofit2/converter/protobuf/ProtoConverterFactory.java
List<ApplicationConfig>.merge() should have reversed priority
Currently, if both configs have the same key, the resulting config will take value from the first one. This can be unintuitive and against common pattern of merging.
To avoid breaking API, this method should be deprecated and two new should be introduced:
public fun ApplicationConfig.mergeWith(other: ApplicationConfig): ApplicationConfig // second has priority
public fun ApplicationConfig.withFallback(other: ApplicationConfig): ApplicationConfig // first has priority
"POSIX error 56: Socket is already connected" error when a socket is connection-mode on Darwin targets
I faced with an error when i'm trying to send Datagram Packet using ktor-network library. I'm not sure if that error reproduced for all nix targets, but it falls for me on macosX64 and all iOS targets. JVM target works fine.
Here is a sample project with tests that reproduce the error https://github.com/KamiSempai/ktor_network_problem.
Test DnsReaderTest.testReadDns
fall with following stack trace:
io.ktor.utils.io.errors.PosixException.PosixErrnoException: POSIX error 56: Socket is already connected (56)
at kotlin.Throwable#<init>(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Throwable.kt:24)
at kotlin.Exception#<init>(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:23)
at io.ktor.utils.io.errors.PosixException#<init>(/opt/buildAgent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/errors/PosixErrors.kt:33)
at io.ktor.utils.io.errors.PosixException.PosixErrnoException#<init>(/opt/buildAgent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/errors/PosixErrors.kt:64)
at io.ktor.utils.io.errors.PosixException.Companion#forErrno(/opt/buildAgent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/errors/PosixErrors.kt:111)
at io.ktor.utils.io.errors.PosixException.Companion#forErrno$default(/opt/buildAgent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/errors/PosixErrors.kt:75)
at io.ktor.network.sockets.DatagramSendChannel.$sendImplCOROUTINE$7.invokeSuspend#internal(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/nix/src/io/ktor/network/sockets/DatagramSendChannel.kt:126)
at io.ktor.network.sockets.DatagramSendChannel.sendImpl#internal(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/nix/src/io/ktor/network/sockets/DatagramSendChannel.kt:106)
at io.ktor.network.sockets.DatagramSendChannel#sendImpl$default(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/nix/src/io/ktor/network/sockets/DatagramSendChannel.kt:106)
at io.ktor.network.sockets.DatagramSendChannel.$sendCOROUTINE$6#invokeSuspend(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/nix/src/io/ktor/network/sockets/DatagramSendChannel.kt:101)
at io.ktor.network.sockets.DatagramSendChannel#send(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/nix/src/io/ktor/network/sockets/DatagramSendChannel.kt:99)
at io.ktor.network.sockets.DatagramWriteChannel#send(/opt/buildAgent/work/8d547b974a7be21f/ktor-network/jvmAndNix/src/io/ktor/network/sockets/Datagram.kt:41)
at com.mytest.DnsReaderTest.$readDnsRecordsCOROUTINE$0.invokeSuspend#internal(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:54)
at com.mytest.DnsReaderTest.readDnsRecords#internal(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:43)
at com.mytest.DnsReaderTest.$testReadDns$lambda-3COROUTINE$2.invokeSuspend#internal(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:35)
at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:30)
at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:106)
at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:284)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:83)
at kotlinx.coroutines#runBlocking(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:56)
at kotlinx.coroutines#runBlocking$default(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:36)
at com.mytest.DnsReaderTest#testReadDns(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:33)
at com.mytest.$DnsReaderTest$test$0.$testReadDns$FUNCTION_REFERENCE$4.invoke#internal(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:32)
at com.mytest.$DnsReaderTest$test$0.$testReadDns$FUNCTION_REFERENCE$4.$<bridge-UNNN>invoke(/Users/denis.shurygin/Documents/AndroidStudioProjects/ktor_network_problem/app/src/commonTest/kotlin/com/mytest/DnsReaderTest.kt:32)
at kotlin.native.internal.test.BaseClassSuite.TestCase#run(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/TestSuite.kt:85)
at kotlin.native.internal.test.TestRunner.run#internal(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:245)
at kotlin.native.internal.test.TestRunner.runIteration#internal(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:271)
at kotlin.native.internal.test.TestRunner#run(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:286)
at kotlin.native.internal.test#testLauncherEntryPoint(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:30)
at kotlin.native.internal.test#main(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:34)
at <global>.Konan_start(/opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:33)
at <global>.Init_and_run_start(Unknown Source)
at <global>.0x0(Unknown Source)
at <global>.0x0(Unknown Source)
Kotlin version: 1.7.10
Ktor version: 2.1.1
Coroutines version: 1.6.4
IDE: Android Studio 2021.2.1 Patch 1 and IntelliJ IDEA 2021.1.1 (Community Edition)
Add Debug Logging to Default Transformers
Add Env Variable to Change Log Level on Native Server
Add Debug Logging to Routing
Add Debug Logging to Auth Plugin
Add Debug Logging to Status Pages Plugin
Add Debug Logging to PartialContent Plugin
Add Debug Logging to Sessions Plugin
Add Debug Logging to Call Id
Add Debug Logging to Double Receive Plugin
Make certificate generation helpers more flexible
Motivation
The initial issue I have with the current state of this library is that the allowed domains are hardcoded (localhost and 127.0.0.1), thus preventing me from allowing hosts like host.docker.internal or 0.0.0.0 or potentially others when generating test certificates for a docker registry mock.
Also, plenty of things are hardcoded in these certificate generation helpers, and there is a lot of code duplication. So I figured I could also polish a bit the API and deduplicate things while I'm at it.
Solution
The point of this PR is to allow for more customizability of the certificate/keystore builder to solve the initial problem. Eventually, the builder became enough to replace most of the bodies of the "all-in-one" generateCertificate functions. Those functions could be deprecated in favor of the more flexible buildKeyStore API (which is now concise enough to rival with those functions), but I wanted to keep this PR focused as it's already broad enough.
Since more properties are exposed, I also made sure to use standard Kotlin or Java types whenever possible, for instance using X500Principal instead of a custom class to represent distinguished names. Speaking of X500Principal, I also wrote a builder for it but it's not part of this PR to avoid an excessive amount of code. If you believe this is interesting to have in the lib too, I can make a separate PR for this.
Because there are a lot of changes which include refactoring, I first wrote a bunch of tests to cover all the existing behaviour and ensure I didn't break anything.
Every commit stands on its own and is quite focused, so don't hesitate to review commit-by-commit. The tests and .api changes are updated after each commit that requires it, so they can be manipulated independently.
Add Debug Logging to Compression Plugin
Make sessions plugin multiplatform
Make API to Use Configuration in Application Plugins
We need to allow using configuration inside application plugins and will start with the simplest way:
class PluginConfig(config: ApplicationConfig) {
val property = config...
}
val myPlugin = createApplicationPlugin("MyPlugin", "ktor.myplugin", ::PluginConfig) {
...
}
This will enable to read configuration file from the specific plugin key.