To reproduce run the following code:

val client = HttpClient(Apache) {
    install(ContentNegotiation) {
        json()
    }
}

val response = client.post("https://httpbin.org/post") {
    contentType(ContentType.Application.Json)
    setBody(123)
}.bodyAsText()

println(response)

I expect a request Content-Type header to be application/json (like in 1.6.7) but actually it's application/json; charset=UTF-8.
The changed behavior unnecessary breaks users' applications after migration to 2.0.0 (https://kotlinlang.slack.com/archives/C0A974TJ9/p1644027042441219).

Issue

If the HttpTimeout plugin is used in parallel with the HttpRequestRetry plugin the connectTimeoutMillis configuration is not respected and the request enters the retry loop.
If the HttpTimeout plugin is installed after HttpRequestRetry the requestTimeoutMillis configuration is not respected as well.

Reproducing the problem

Here you can find a basic example that reproduces the problem.

Platform

Noticed on JVM

Hello Ktor team :)

It has been pleasure to work with Ktor so far, but we stumbled on some issue that we are not sure how to solve, I will try to provide as much information as I can but if you need anything else from me please let me know so that perhaps we can figure something out together :)

We are trying to implement the feature where we want to cancel an outgoing asset upload to the server, this is basicly what we are trying to achieve :

class StreamAssetContent(
    private val fileContentStream: okio.Source
) : OutgoingContent.WriteChannelContent()  {

    override suspend fun writeTo(channel: ByteWriteChannel) {
        val contentBuffer = Buffer()
        while (fileContentStream.read(contentBuffer, BUFFER_SIZE) != -1L) {
            contentBuffer.readByteArray().let { content ->
                channel.writePacket(ByteReadPacket(content))
            }
        }
        channel.writeStringUtf8(closingArray)
        channel.flush()
        channel.close()
    }

    private companion object {
        const val BUFFER_SIZE = 1024 * 8L
    }
}

suspend fun cancelDuringUpload() {
    val httpClient = HttpClient(engine = OkHttp.create())
    
    val uploadJob = launch {
        httpClient.post(PATH_PUBLIC_ASSETS_V3) {
            contentType(ContentType.MultiPart.Mixed)
            setBody(StreamAssetContent(fileSource))
        }
    }
    
    delay(100.milliseconds)
    uploadJob.cancel()
}

according to my knowledge this should work as it cancels the coroutine it is launched in, but it crashes the app with CancellationException, we were able to solve
by isolating the throwing of the CancellationException from the insides of the Ktor

class StreamAssetContent(
    private val fileContentStream: okio.Source
) : OutgoingContent.WriteChannelContent()  {
   override suspend fun writeTo(channel: ByteWriteChannel) {
        try {
       coroutineScope {
                if (!channel.isClosedForWrite && producerJob.isActive) {
                    channel.writeStringUtf8(openingData)
                    val contentBuffer = Buffer()
                    while (fileContentStream.read(contentBuffer, BUFFER_SIZE) != -1L) {
                        contentBuffer.readByteArray().let { content ->
                            channel.writePacket(ByteReadPacket(content))
                        }
                    }
                    channel.writeStringUtf8(closingArray)
                    channel.flush()
                    channel.close()
                }
            }
        } catch (e: Exception) {
            channel.flush()
            channel.close()
            producerJob.completeExceptionally(e)
            throw IOException(e.message)
        } finally {
            producerJob.complete()
        }
    }
}

This would be a nice fix, but unfortunetly this was not enough, because when the timing is right the content of this class coming from Ktor also can throw CancellationException

internal class StreamRequestBody(
    private val contentLength: Long?,
  private val block: () -> ByteReadChannel`
) : RequestBody() {
    override fun contentType(): MediaType? = null
    override fun writeTo(sink: BufferedSink) {
        block().toInputStream().source().use {
            sink.writeAll(it)
       }
    }
    override fun contentLength(): Long = contentLength ?: -1
    override fun isOneShot(): Boolean = true
}

so we again override it and this is we come up with :

class ByteChannelRequestBody(
    private val contentLength: Long?,
    private val callContext: CoroutineContext,
    private val block: () -> ByteReadChannel
) : RequestBody(), CoroutineScope {
    private val producerJob = Job(callContext[Job])

    override val coroutineContext: CoroutineContext
        get() = callContext + producerJob + Dispatchers.IO
    override fun contentLength(): Long = contentLength ?: -1
    override fun contentType(): MediaType? = null

    override fun writeTo(sink: BufferedSink) {
        withJob(producerJob) {
            if (producerJob.isActive) {
                block().toInputStream().source().use {
                    sink.writeAll(it)
                }
            }
        }
    }

    private inline fun <T> withJob(job: CompletableJob, block: () -> T): T {
        try {
            return block()
        } catch (ex: Exception) {
           job.completeExceptionally(ex)
            // wrap all exceptions thrown from inside `okhttp3.RequestBody#writeTo(..)` as an IOException`    
            throw IOException(ex)
        } finally {
            job.complete()
        }
    }
}

the issue here is that in order to use this imlementation I had to copy-paste Ktor source code, because the place that is allowing us to use our custom Class is Ktor internal. This requires us to maintain the Ktor classes with each update, which we would ike not to do since you guys are doing it way better :P

internal fun OutgoingContent.convertToOkHttpBody(callContext: CoroutineContext): RequestBody = when (this) {
    is OutgoingContent.ByteArrayContent -> bytes().let {
        it.toRequestBody(null, 0, it.size)
    }
    is OutgoingContent.ReadChannelContent -> ByteChannelRequestBody(contentLength, callContext) { readFrom() }
    is OutgoingContent.WriteChannelContent -> {
        ByteChannelRequestBody(contentLength, callContext) {
            GlobalScope.writer(callContext) { writeTo(channel) }.channel
        }
    }
    is OutgoingContent.NoContent -> ByteArray(0).toRequestBody(null, 0, 0)
    else -> throw UnsupportedContentTypeException(this)
}

We are on :

okio = 3.2.0
ok-http = 4.9.3
ktor = 2.2.1

Here is also the stacktrace when the issues are happening:

com.wire.kalium.network.exceptions.KaliumException$GenericError
	at com.wire.kalium.network.api.v2.authenticated.AssetApiV2.uploadAsset$suspendImpl(AssetApiV2.kt:89)
	at com.wire.kalium.network.api.v2.authenticated.AssetApiV2$uploadAsset$1.invokeSuspend(Unknown Source:18)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.util.concurrent.CancellationException: test
	at com.wire.kalium.network.http.request.ByteChannelRequestBody.writeTo(ByteChannelRequestBody.kt:63)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:59)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
	at java.lang.Thread.run(Thread.java:1012)

so my question to you, is there anything we are perhaps doing wrong, or is this simply not yet easily supported in Ktor ?

There is also an issue on this from AWS :
https://github.com/awslabs/aws-sdk-kotlin/issues/733

And this is the PR coming from us - Wire
https://github.com/wireapp/kalium/pull/1288

thanks for any help and let us know if you need anything else from us

Unable to use underscores in request URL with Ktor http client. Client fails to parse port and name correctly urls with underscores.
For example http://my_service:8080 fails to parse correctly. Calling URL("http://my_service:8080") on seems to parse port and hostname but calling toURI()on it for seems to fail to parse port and hostname correcly. This is an issue with URLBuilder.takeFrom(url: URL): URLBuilder = takeFrom(url.toURI()) This seems to be somewhat known issue with Java URI class.

Hostnames with underscores are quite common in docker environments and crucial for my use case. Is this something you would be looking to support?

Link to related method: https://github.com/ktorio/ktor/blob/0081f943b434bdd0afd82424b389629e89a89461/ktor-http/jvm/src/io/ktor/http/URLUtilsJvm.kt#L49

The Plugin should be listed in the Generator as a MongoDB ${used library} plugin with some short description and code snippet. On apply it should add:

• required dependencies
• simple database model with initialization
• 4 CRUD endpoints

We have a problem after releasing a new version of Ktor; there is a time gap between the release of Ktor and the Ktor Gradle plugin, so a new project with the latest released version cannot be built because of the absence of the respective version of Ktor Gradle plugin.

The Plugin should be listed in the Generator as a Postgres plugin with some short description and code snippet. On apply it should add:

• required dependencies
• simple database model with initialization
• 4 CRUD endpoints

The Plugin should be listed in the Generator as an Exposed plugin with some short description and code snippet. On apply it should add:

• expose dependencies
• simple database model with initialization
• 4 CRUD endpoints

1. Type

fun Application.configureRouting() {
    install(StatusPages) {
        exception<IllegalStateException> { call, cause ->
            call.respond(HttpStatusCode.BadGateway, "This is response")
        }
    }
    routing {
        get("/") {
            call.respondText("Hello World!")
        }
        get("/{name}") {
            throw IllegalStateException("FOO")
        }
    }
}

2. Generate OpenAPI
3. Remove exception handler:

fun Application.configureRouting() {
    install(StatusPages) {
    }
    routing {
        get("/") {
            call.respondText("Hello World!")
        }
        get("/{name}") {
            throw IllegalStateException("FOO")
        }
    }
}

4. Generate OpenAPI.

Expected: Now OpenAPI for /{name} does not reflect response HttpStatusCode.BadGateway, "This is response" now.
Actual: It hasn't changed.

    at io.ktor.ide.KtorRoutingVisitor.processLastArgumentsBodyIfRouteExtensionLambda(KtorUrlResolver.kt:149)
    at io.ktor.ide.KtorRoutingVisitor.visitCallExpression(KtorUrlResolver.kt:188)
    at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:165)
    at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
    at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
    at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
    at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
    at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:169)
    at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
    at org.jetbrains.uast.UBlockExpression.accept(UBlockExpression.kt:21)
    at org.jetbrains.uast.ULambdaExpression.accept(ULambdaExpression.kt:40)
    at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
    at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:169)
    at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings$lambda$3$lambda$2(KtorUrlResolver.kt:65)
    at io.ktor.ide.KtorUtilsKt.withProgress(KtorUtils.kt:112)
    at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings$lambda$3(KtorUrlResolver.kt:64)
    at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:39)
    at com.intellij.util.CachedValueBase.lambda$getValueWithLock$3(CachedValueBase.java:231)
    at com.intellij.util.CachedValueBase.computeData(CachedValueBase.java:41)
    at com.intellij.util.CachedValueBase.lambda$getValueWithLock$4(CachedValueBase.java:231)
    at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:112)
    at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
    at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:66)
    at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:232)
    at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:28)
    at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:72)
    at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:111)
    at io.ktor.ide.KtorUrlResolverKt.getAllUrlMappings(KtorUrlResolver.kt:61)
    at io.ktor.ide.KtorUrlResolverKt.access$getAllUrlMappings(KtorUrlResolver.kt:1)
    at io.ktor.ide.KtorUrlResolver$getVariants$2.invoke(KtorUrlResolver.kt:41)
    at io.ktor.ide.KtorUrlResolver$getVariants$2.invoke(KtorUrlResolver.kt:41)
    at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:315)
    at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)
    at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
    at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
    at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
    at kotlin.sequences.FilteringSequence$iterator$1.calcNext(Sequences.kt:169)
    at kotlin.sequences.FilteringSequence$iterator$1.hasNext(Sequences.kt:194)
    at com.intellij.microservices.gotosymbol.UrlSearchEverywhereContributor$fetchWeightedElements$lambda$2$$inlined$computeInReadActionWithWriteActionPriority$1.run(UrlSearchEverywhereContributor.kt:205)
    at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1086)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:71)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtilService.runActionAndCancelBeforeWrite(ProgressIndicatorUtilService.java:63)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:128)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:109)
    at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:68)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:589)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:664)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:620)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:588)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
    at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:68)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:106)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:71)
    at com.intellij.microservices.gotosymbol.UrlSearchEverywhereContributor.p(UrlSearchEverywhereContributor.kt:202)
    at com.intellij.concurrency.JobLauncherImpl.lambda$processImmediatelyIfTooFew$2(JobLauncherImpl.java:137)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:589)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:664)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:620)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:588)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
    at com.intellij.concurrency.JobLauncherImpl.lambda$processImmediatelyIfTooFew$3(JobLauncherImpl.java:133)
    at com.intellij.concurrency.JobLauncherImpl.processImmediatelyIfTooFew(JobLauncherImpl.java:147)
    at com.intellij.concurrency.JobLauncherImpl.invokeConcurrentlyUnderProgress(JobLauncherImpl.java:44)

232.1708

fun Application.configureRouting() {
    install(StatusPages) {
        exception<IllegalStateException> { call, cause ->
            call.respond(HttpStatusCode.BadGateway, "500")
        }
    }
    routing {
        get("/") {
            call.respondText("Hello World!")
        }
        get("/{name}") {
            throw IllegalStateException("FOO")
//            call.respond("OK")
//            val name = call.parameters["name"] ?:
//            call.respondText("Hello $name!")
        }
    }
}

Exception:

java.lang.IllegalStateException: Calling invokeAndWait from read-action leads to possible deadlock.
    at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:454)
    at com.intellij.openapi.application.ex.ApplicationUtil.invokeAndWaitSomewhere(ApplicationUtil.java:145)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcessWithProgressSynchronously(CoreProgressManager.java:532)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.runProcessWithProgressSynchronously(ProgressManagerImpl.java:85)
    at com.intellij.openapi.progress.impl.CoreProgressManager.run(CoreProgressManager.java:367)
    at com.intellij.openapi.progress.ProgressManager.run(ProgressManager.java:208)
    at io.ktor.ide.actions.test.TestActionsUtils$Companion.executeWithModalProgress(TestActionsUtils.kt:65)
    at io.ktor.ide.actions.test.TestActionsUtils$Companion.findParentRoutingCall$intellij_ktor_starter(TestActionsUtils.kt:213)
    at io.ktor.ide.actions.test.TestActionsUtils$Companion.findKtorApplicationConfig(TestActionsUtils.kt:116)
    at io.ktor.ide.oas.KtorStatusPagesOasSupportKt.findStatusPagesMapping(KtorStatusPagesOasSupport.kt:22)
    at io.ktor.ide.oas.KtorOasVisitor.findStatusPagesMappingCached$lambda$6(KtorOasVisitor.kt:120)

Ex:

install(StatusPages) {
    exception<Throwable> { call, cause ->
        call.respondText(text = "500: ${'$'}cause", status = HttpStatusCode.InternalServerError)
    }
    exception<IllegalStateException> { call, cause ->
        call.respond("AAA")
    }
}

...

get("/eee") {
    error("aaa")
    myError()
}

...      

fun myError(): Nothing = throw Exception()

In this case, /eee route in OpenAPI does not represent handlers from StatusPages because myError is recursive call that throws (although it is not Route. or Call. extension)

https://kotlinlang.slack.com/archives/C0A974TJ9/p1680268685834439

If the Ktor Gradle plugin is applied to the project, the specific versions for Ktor's dependencies are redundant.

In continuation of KTOR-4849. The example provided there is indeed fixed in 2.2.3. However, if we add another route, another issue emerges. Instead of 415 I get 405.

package com.example

import io.ktor.http.ContentType
import io.ktor.http.HttpMethod
import io.ktor.server.application.Application
import io.ktor.server.application.call
import io.ktor.server.application.log
import io.ktor.server.engine.embeddedServer
import io.ktor.server.netty.Netty
import io.ktor.server.response.respond
import io.ktor.server.routing.accept
import io.ktor.server.routing.contentType
import io.ktor.server.routing.route
import io.ktor.server.routing.routing

fun main() {
    embeddedServer(Netty, port = 3000) {
        module()
    }.apply {
        start(wait = true)
    }
}

fun Application.module(): Unit {
    routing {
        route("/some/plain/path", HttpMethod.Post) {
            contentType(ContentType.Application.Json) {
                handle {
                    call.respond("ok")
                }
            }
        }
        route("/some/plain/path", HttpMethod.Get) {
            contentType(ContentType.Application.Json) {
                handle {
                    call.respond("ok")
                }
            }
        }
    }
}

$ curl -i -XPOST localhost:3000/some/plain/path -H'Content-Type: application/xml' 
HTTP/1.1 405 Method Not Allowed
Content-Length: 0

sample code

• /compress returns a compressed response.
• /normal returns a not compressed response.

fun Application.module() {
    val logger = LoggerFactory.getLogger(Application::class.java)
    install(Compression) {
        condition {
            request.uri.startsWith("/compress")
        }
    }
    install(StatusPages) {
        exception<IllegalStateException> { call, cause ->
            logger.info("error! request uri: {}", call.request.uri)
            call.respond(HttpStatusCode.BadRequest, "400error!!!!")
        }
    }
    routing {
        get("/compress") { throw IllegalStateException() }
        get("/normal") { throw IllegalStateException() }
    }
}

result(log output)

Compressing the response produces an unexpected ERROR log.

2023-02-01 18:10:33.526 [eventLoopGroupProxy-4-1] INFO  i.k.server.application.Application - error! request uri: /normal
2023-02-01 18:10:41.711 [eventLoopGroupProxy-4-1] INFO  i.k.server.application.Application - error! request uri: /compress
2023-02-01 18:10:41.736 [eventLoopGroupProxy-4-1] ERROR ktor.application - 400 Bad Request: GET - /compress

Cause

Could the following code be the cause? The response is not changed to sent.

https://github.com/ktorio/ktor/blob/f97c1a3432041471c1ae3eed2ecc9047c8ad44a7/ktor-server/ktor-server-host-common/jvmAndNix/src/io/ktor/server/engine/BaseApplicationResponse.kt#L136-L146

The exception is resent because the response has not been sent.
https://github.com/ktorio/ktor/blob/f97c1a3432041471c1ae3eed2ecc9047c8ad44a7/ktor-server/ktor-server-core/jvmAndNix/src/io/ktor/server/application/hooks/CommonHooks.kt#L40-L49

The Javadoc for Resources.kt shows the following example, which is wrong (the Users data class must have at least one primary constructor parameter). So the best fix would probably be to switch out the data classes with regular classes.

I wouldn't mind opening a pull request btw.

/**
 * Adds support for type-safe routing using [ResourcesCore].
 *
 * Example:
 * ```kotlin
 * @Serializable
 * @Resource("/users")
 * data class Users {
 *   @Serializable
 *   @Resource("/{id}")
 *   data class ById(val parent: Users = Users(), val id: Long)
 *
 *   @Serializable
 *   @Resource("/add")
 *   data class Add(val parent: Users = Users(), val name: String)
 * }

Consider this class:

@Serializable
class Foo<T>(val f: Float, val t: T?)

And this test that uses kotlinx.serialization.json.Json { allowSpecialFloatingPointValues = false } (other test infrastructure taken from Ktor's JsonClientKotlinxSerializationTest):

 @Test fun testGenericFloat(): Unit = testWithEngine(CIO) {
        configureClient()

        test { client ->
            val result = client.post {
                url(path = "/echo", port = serverPort)
                contentType(defaultContentType)
                setBody(Foo<String>(Float.NEGATIVE_INFINITY, "f"))
            }.body<Foo<String>>()
        }
    }

The problem is following: normally, Json.encodeToString can throw arbitrary SerializationException to indicate problems with input data - e.g. invalid floating point value. It is not the only exception we can throw, but it is very simple to be illustrative.
However, Ktor apparently swallows this exception in this point: https://github.com/ktorio/ktor/blob/bad1c2da472646318a402342a249cc6300bc0940/ktor-shared/ktor-serialization/ktor-serialization-kotlinx/common/src/io/ktor/serialization/kotlinx/KotlinxSerializationBase.kt#L29

There are several problems with code:

1. Ktor assumes that only SerializationException we can get is 'serializer not found', but it is not true.
2. Exception message or stack trace is lost.
3. Instead, we proceed to do guessSerializer (https://github.com/ktorio/ktor/blob/bad1c2da472646318a402342a249cc6300bc0940/ktor-shared/ktor-serialization/ktor-serialization-kotlinx/common/src/io/ktor/serialization/kotlinx/KotlinxSerializationBase.kt#L34) which don't support generic classes. As a result, there is a cryptic message after running the test: Serializer for class 'Foo' is not found. Mark the class as @Serializable or provide the serializer explicitly., while it is clear that Foo is @Serializable.

Given that mentioned https://github.com/Kotlin/kotlinx.serialization/issues/1163 is already fixed, I suggest rewriting this place

What would it mean
i.k.s.p.c.ContentNegotiation - Skipping because the type is ignored.
This message is unclear

IU-223.8214.52, JRE 17.0.5+1-b653.23x64 JetBrains s.r.o., OS Windows 10(amd64) v10.0 , screens 1920.0x1080.0, 1920.0x1080.0

REQUEST https://api.github.com/repos/Example/Repo/compare/eap...GL-1412-folder-with-multi-repos-in-it-the-you-are-offline-footer-stays-+-doesnt failed with exception: java.io.FileNotFoundException: /Users/username/Library/Caches/AppName/68747470733a2f2f6170692e6769746875622e636f6d2f7265706f732f4769744c6976654170702f636f72652f636f6d706172652f6561702e2e2e474c2d313431322d666f6c6465722d776974682d6d756c74692d7265706f732d696e2d69742d7468652d796f752d6172652d6f66666c696e652d666f6f7465722d73746179732d2b2d646f65736e74 (File name too long)

Environment

• Kotlin version: 1.8
• Library version: 1.4.1
• Kotlin platforms: JVM
• Gradle version: 7.4.1
• OS Version: macOS 12.5

With a configuration that includes HttpCache and HttpRequestRetry plugins such as:

        HttpClient {
            expectSuccess = true
            install(HttpCache) { privateStorage(FileStorage(Files.createDirectories(Paths.get(dirs.cacheDir)).toFile())) }
            install(ContentNegotiation) { json(json) }
            Logging {
                level = LogLevel.INFO
            }
            install(HttpRequestRetry) {
                maxRetries = 5
            }

Exceptions thrown by HttpCache such as FileNotFoundException (see KTOR-5443) cause HttpRequestRetry to retry the request and hit the cache again. I would expect the behavior to be that any exceptions thrown accessing the cache do not fail the request but instead continue the request as if there was a cache miss.

1. Remove from https://ktor.io/learn/
2. Redeploy the site
3. Update TeamCity configuration
4. Remove from the repo: https://github.com/ktorio/ktor-samples

To reproduce, generate a project with the Exposed plugin. As a result, the build fails with the Overload resolution ambiguity error:

public final val h2_version: String defined in Build_gradle
public final val h2_version: String defined in Build_gradle

To reproduce, create a new project with Ktor 2.1.3 through the wizard with the Gson plugin and run "Migrate Project to Latest Ktor Version...".
As a result, the correct import statement import io.ktor.serialization.gson.* is replaced with the incorrect one import io.ktor.serialization.kotlinx.json.gson.* in the plugins/Serialization.kt file.

Example:

@Resource("/v1")
class V1Api {
    @Resource("/reports")
    class Reports(val parent: V1Api) {
        @Resource("/")
        class New(val parent: Reports)

        @Resource("{id}")
        class Id(val parent: Reports, val id: Int) {
            @Resource("/timeline")
            class Timeline(val parent: Id)
        }
    }
}

routing {
    post<V1Api.Reports.New> {
        call.respondText("hello")
    }
}

Resulting schema contains:

post:
      description: ""
      parameters:
      - name: "parent"
        in: "query"
        required: true
        schema:
          type: "object"
      - name: "parent"
        in: "query"
        required: true
        schema:
          type: "object"

While parent fields should not be considered as query parameters, but as a "technical" references to the parent class in the code (see https://ktor.io/docs/type-safe-routing.html#resource_nested)

Steps to reproduce:

• check out the reproducer project. It contains a simple upload sample and Gradle run task that constrains memory to 128mb.
• Create a large file to upload (e.g. on macOS with mkfile -n 2g upload.bin)
• Navigate to http://0.0.0.0:8080/upload and upload the file

The significant part seems to be UploadRoute's stream-copying:

                it.streamProvider().use { partstream ->
                    targetFile.outputStream().use { os ->
                        partstream.copyTo(os, 8192)
                    }
                }

It results in the following:

2021-11-21 20:49:35.681 [eventLoopGroupProxy-4-1] ERROR ktor.application - Unhandled exception caught for CoroutineName(call-handler)
kotlinx.coroutines.channels.ClosedReceiveChannelException: Unexpected EOF: expected 4096 more bytes
	at io.ktor.utils.io.ByteBufferChannel.readFullySuspend(ByteBufferChannel.kt:573)
	at io.ktor.utils.io.ByteBufferChannel.readFully(ByteBufferChannel.kt:565)
	at io.ktor.utils.io.ByteBufferChannel.readPacketSuspend(ByteBufferChannel.kt:781)
	at io.ktor.utils.io.ByteBufferChannel.readPacket$suspendImpl(ByteBufferChannel.kt:767)
	at io.ktor.utils.io.ByteBufferChannel.readPacket(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteReadChannelKt.readPacket(ByteReadChannel.kt:198)
	at io.ktor.http.cio.MultipartKt$parseMultipart$1.invokeSuspend(Multipart.kt:342)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:205)
	at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
	at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2138)
	at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:887)
	at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:483)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:678)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteBufferChannel.readAvailableSuspend(ByteBufferChannel.kt:722)
	at io.ktor.utils.io.ByteBufferChannel.access$readAvailableSuspend(ByteBufferChannel.kt:24)
	at io.ktor.utils.io.ByteBufferChannel$readAvailableSuspend$2.invokeSuspend(ByteBufferChannel.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:260)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:831)

My two-gigabyte sample files have only arrived in part to /tmp, as well.

Event Routing.RoutingCallFinished happens before hook CallFailed.

Because of this all the responses that were handled by the StatusPages plugin has incorrect metrics status 0.

DropwizardMetrics.kt

     on(MonitoringEvent(Routing.RoutingCallFinished)) { call ->
            val routingMetrics = call.attributes.take(routingMetricsKey)
            val status = call.response.status()?.value ?: 0
            val statusMeter =
                pluginConfig.registry.meter(name(pluginConfig.baseName, routingMetrics.name, status.toString()))
            statusMeter.mark()
            routingMetrics.context.stop()
        }

StatusPages.kt

    on(CallFailed) { call, cause ->
        if (call.attributes.contains(statusPageMarker)) return@on

        LOGGER.trace("Call ${call.request.uri} failed with cause $cause")

        val handler = findHandlerByValue(cause)
        if (handler == null) {
            LOGGER.trace("No handler found for exception: $cause for call ${call.request.uri}")
            throw cause
        }

        call.attributes.put(statusPageMarker, Unit)
        call.application.mdcProvider.withMDCBlock(call) {
            LOGGER.trace("Executing $handler for exception $cause for call ${call.request.uri}")
            handler(call, cause)
        }
    }

In the documentation there is an example that uses ResponseSent hook which seems to me more suitable for such case https://ktor.io/docs/events.html#custom-events

Is there any workaround, or should the DropwizardMetricsPlugin be modified to subscribe to another hook?

I also saw a discussion here to back up this idea https://github.com/ktorio/ktor/pull/2792#discussion_r791846131 however this branch wasn't merged to main branch afaics.

I installed CallLogging on my Ktor server and it works great to keep track of what's going on!

I was wondering if there is a way to also log the ms it took to process the request, as of now it looks like this:
2020-11-05 14:12:04.248 [ktor-jetty-8080-5] INFO Application - 200 OK: GET - /users/23

Hi, team!

Problem:

I'm trying to set tcp keep-alive settings for our ktor netty server:

embeddedServer(Netty, port = Configuration.appPort, configure = {
    tcpKeepAlive = true
    ...
  }
  ...
)

However, netty warns me that SO_KEEPALIVE is an unknown option: {"timestamp":"2022-12-21 17:27:23.982","level":"WARN","thread":"main","logger":"io.netty.bootstrap.ServerBootstrap","message":"Unknown channel option 'SO_KEEPALIVE' for channel '[id: 0x0bdc9e8b]'","context":"default"}

Workaround:

The problem could be solved with this workaround:

embeddedServer(Netty, port = Configuration.appPort, configure = {
    configureBootstrap = {
        childOption(ChannelOption.SO_KEEPALIVE, true)
    }
    ...
  }
  ...
)

Possible fix:

So, it looks like, you are setting SO_KEEPALIVE option in a wrong way:

if (configuration.tcpKeepAlive) {
  option(NioChannelOption.SO_KEEPALIVE, true)
}

Details:

    implementation("io.ktor:ktor-server-netty:2.0.1")
    implementation("io.netty:netty-transport-native-epoll:4.1.77.Final:linux-x86_64")
    implementation("io.netty:netty-transport-native-kqueue:4.1.77.Final:osx-x86_64")

Even after fixing https://youtrack.jetbrains.com/issue/KTOR-5000 CLI parameters not overriding custom array properties.

application.conf example:

ktor {
  deployment {
    port = 8085
  }
}
some {
  custom {
    arrProp = [ 
      pumpurum,
      // ololo,
    ]
  }
}

Command example:

java -jar build/libs/app.jar -P:some.custom.arrProp.0=azaza -P:some.custom.arrProp.1=ololo -P:some.custom.arrProp.2=ururu

Expected bevaviour:

val n = config.config("some.custom").property("arrProp").getList()
// n is ["azaza", "ololo", "ururu"] (from command line)

Actual behaviour:

val n = config.config("some.custom").property("arrProp").getList()
// n is ["pumpurum"] (default value from file) 

Example. When user calls

val report = call.receive<Report>()

There should be the following entry in OpenAPI:

  requestBody:
    content:
      '*/*':
        schema:
          $ref: "#/components/schemas/Report"
    required: true

Summary

OAuth2 functionality is currently JVM-only code under ktor-features/ktor-auth/.
But I'm wondering if this should be a common client feature and be ported to ktor-client-features/ktor-client-auth/

Environment

Ktor version: 1.4.1
Subsystem: Client
Engine: OkHttp

Details

Perhaps it is my lack of understanding or perhaps it's already a tech-debt item, but I don't think there is anything JVM-specific in the implementatoin of the OAuth2 authorization flows in OAuth2.kt.

On Android/JVM, I successfully used the verifyWithOAuth2 method to retrieve an access token. However, my ultimate goal is doing this on Windows / mingw64.

OAuth2 functionality currently is JVM-only code under
ktor-features/ktor-auth/jvm/src/io/ktor/auth

However, this seems like it should be a common client feature and therefore be ported to
ktor-client-features/ktor-client-auth/common