Changelog 1.4 version

1.4.3

released 1st December 2020

Client

Missing Content-Type header in a request

ClassCastException with HTTP client when using via an inline function with reified type parameter

https://youtrack.jetbrains.com/issue/KTOR-1423

ktor HTTP client results in a ClassCastException ("class java.util.LinkedHashMap cannot be cast to class MyEnvironmentDetails").

suspend fun main() {
    InternalClient().use {
        it.getEnvironment()
    }
}

@KtorExperimentalAPI
class MyClient: Closeable {
    private val client: HttpClient = HttpClient(CIO) {
        install(JsonFeature) {
            serializer = JacksonSerializer {
                registerModule(JavaTimeModule())
            }
        }
    }

    suspend fun getEnvironment(): MyEnvironmentDetails {
        return internalRequest("Core.GetEnvironment")
    }

    private suspend inline fun <reified T> internalRequest(method: String): T {
        val rawResponse = client.post<RawApiResponse<T>> {
            url("https", "my.site", 443, "/api")

            accept(ContentType.Application.Json)
            contentType(ContentType.Application.Json)

            body = ApiRequest(method, Unit)
        }

        return rawResponse.outParams
    }

    override fun close() {
        client.close()
    }
}

data class ApiRequest<T>(
    val action: String,
    val inParams: T,
) {
    val apiVersion: BigDecimal = 2.0.toBigDecimal()
}

@JsonIgnoreProperties(ignoreUnknown = true)
data class RawApiResponse<T>(
    val outParams: T,
    val sessionTrackingId: String?,
)

@JsonIgnoreProperties(ignoreUnknown = true)
data class MyEnvironmentDetails(
    val lang: String,
    val title: String,
)

If I manually inline internalRequest(), e.g. so that getEnvironment() becomes:

suspend fun getEnvironment(): MyEnvironmentDetails {
    val rawResponse = client.post<RawApiResponse<MyEnvironmentDetails>> {
        url("https", "my.site", 443, "/api")

        accept(ContentType.Application.Json)
        contentType(ContentType.Application.Json)

        body = ApiRequest("Core.GetEnvironment", Unit)
    }

    return rawResponse.outParams
}

It works as expected (i.e. no ClassCastException). Tracking this down through the debugger I can see that the io.ktor.client.call.TypeInfo<T> in my initial example has a reifiedType that represents a RawApiResponse<T> and the working example has a reifiedType that represents a RawApiResponse< MyEnvironmentDetails>.

Can I logging single line?

iOS testing MockEngine issue

ClassCastException: kotlin.coroutines.intrinsics.CoroutineSingletons cannot be cast to ...

https://youtrack.jetbrains.com/issue/KTOR-2695

Hi there. what is a good way to approach debugging this?
I am using suspend fun as well as Flows and awaits, but I am not sure where to start with this.
on Slack, Roman suggested this might be ktor related.

Exception in thread "main" java.lang.ClassCastException: kotlin.coroutines.intrinsics.CoroutineSingletons cannot be cast to io.ktor.client.statement.HttpResponse
        at com.github.wakingrufus.kalibrate.example.MainKt$main$2$4$1$1$1.invoke(Main.kt:68)
        at com.github.wakingrufus.kalibrate.example.MainKt$main$2$4$1$1$1.invoke(Main.kt)
        at com.github.wakingrufus.kalibrate.scenario.Step.invoke(Step.kt:15)
        at com.github.wakingrufus.kalibrate.scenario.StepContainer$invoke$1.invokeSuspend(Simulation.kt:93)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:277)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:86)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:61)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.github.wakingrufus.kalibrate.scenario.StepContainer.invoke(Simulation.kt:91)
        at com.github.wakingrufus.kalibrate.scenario.Simulation.runSetup(Simulation.kt:75)
        at com.github.wakingrufus.kalibrate.scenario.Simulation$singletonWorkPattern$1$1$1.invokeSuspend(Simulation.kt:25)
        at com.github.wakingrufus.kalibrate.scenario.Simulation$singletonWorkPattern$1$1$1.invoke(Simulation.kt)
        at kotlinx.coroutines.flow.SafeFlow.collectSafely(Builders.kt:61)
        at kotlinx.coroutines.flow.AbstractFlow.collect(Flow.kt:212)
        at kotlinx.coroutines.flow.FlowKt__MergeKt$flattenConcat$$inlined$unsafeFlow$1.collect(SafeCollector.common.kt:114)
        at kotlinx.coroutines.flow.internal.ChannelFlowMerge$collectTo$$inlined$collect$1$lambda$1.invokeSuspend(Merge.kt:69)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:277)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:86)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:61)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.github.wakingrufus.kalibrate.KalibrateDslBuilder.invoke(KalibrateDsl.kt:92)
        at com.github.wakingrufus.kalibrate.KalibrateDslKt$kalibrate$1.invoke(KalibrateDsl.kt:125)
        at com.github.wakingrufus.kalibrate.KalibrateDslKt$kalibrate$1.invoke(KalibrateDsl.kt)
        at com.xenomachina.argparser.SystemExitExceptionKt.mainBody(SystemExitException.kt:74)
        at com.xenomachina.argparser.SystemExitExceptionKt.mainBody$default(SystemExitException.kt:72)
        at com.github.wakingrufus.kalibrate.KalibrateDslKt.kalibrate(KalibrateDsl.kt:124)
        at com.github.wakingrufus.kalibrate.example.MainKt.main(Main.kt:16)

source:
https://github.com/wakingrufus/kalibrate

to reproduce:
cd kalibrate-example/
./example.sh --scenario deploy

'%3D' inside query of redirect target location will be replaced to '='

https://youtrack.jetbrains.com/issue/KTOR-2057

When the server returns 302 with the URL has '%3D' (inside query part), the Ktor replaces '%3D' with '=' so some server returns invalid content (in this case, 403).

Test code:

import io.ktor.client.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.runBlocking

fun main(): Unit = runBlocking {
    val url = "https://dl.bintray.com/anatawa12/maven/com/anatawa12/forge/ForgeGradle/1.2-1.0.0/ForgeGradle-1.2-1.0.0-javadoc.jar"

    // also in CIO
    //val factory = io.ktor.client.engine.cio.CIO
    val factory = io.ktor.client.engine.apache.Apache
    val clientNoFollow = HttpClient(io.ktor.client.engine.cio.CIO) {
        followRedirects = false
        expectSuccess = false
    }
    val response = clientNoFollow.get<HttpResponse>(url)
    println("status: " + response.status)
    println("location:  " + response.headers["Location"])
    println("location has %3D: " + response.headers["Location"]!!.contains("%3D"))
    check(response.headers["Location"]!!.contains("%3D"))

    val clientWithFollow = HttpClient(io.ktor.client.engine.cio.CIO)
    try {
        val got = clientWithFollow.get<ByteArray>(url)
    } catch (e: ResponseException) {
        println("error: " + e)
        println("error url: " + e.response.request.url)
        println("error url has %3D: " + e.response.request.url.toString().contains("%3D"))
    }
}

Log:

status: 302 
location:  https://d29vzk4ow07wi7.cloudfront.net/d133d0ab57f0e1a996fa1367eb3e5c7ebbd1860a5d505a69791d7746457acb0a?response-content-disposition=attachment%3Bfilename%3D%22ForgeGradle-1.2-1.0.0-javadoc.jar%22&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHAqOi8vZDI5dnprNG93MDd3aTcuY2xvdWRmcm9udC5uZXQvZDEzM2QwYWI1N2YwZTFhOTk2ZmExMzY3ZWIzZTVjN2ViYmQxODYwYTVkNTA1YTY5NzkxZDc3NDY0NTdhY2IwYT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPWF0dGFjaG1lbnQlM0JmaWxlbmFtZSUzRCUyMkZvcmdlR3JhZGxlLTEuMi0xLjAuMC1qYXZhZG9jLmphciUyMiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTYxMjkzNzUyOX0sIklwQWRkcmVzcyI6eyJBV1M6U291cmNlSXAiOiIwLjAuMC4wLzAifX19XX0_&Signature=ch~WD7z9cBluu3KWX2pIsmCd7pfVQ~WnM2QEg6wLuDPFi5pr2e4DnHApdAX9A92dxJW68WYllbKB1pNRkJ6RjtW~3eip545DnUcSBYGT1vfiaxNzp80QYsg~EUo~l4eHFYVoywXXPNgEPRXPIZ2vHGv7H7aeGJ5fbhzyunSmdstRnfUpfKLL5XmsDjpvVY4RYy1NSHHG70BU3y6UP1nqdu1hEdzBhRFKwZ~0uj8~2S4OF6Daaj71jkRWF49TQS~aNpvrkaAZojUilWEwppXZNcCekXu7Bt95oJDESsNQf0bu0gDw5eNtAfy2ZzN~a~lXVTV3ORy3kVanNwiVJzcf~g__&Key-Pair-Id=APKAIFKFWOMXM2UMTSFA
location has %3D: true
error: io.ktor.client.features.ClientRequestException: Client request(https://d29vzk4ow07wi7.cloudfront.net/d133d0ab57f0e1a996fa1367eb3e5c7ebbd1860a5d505a69791d7746457acb0a?response-content-disposition=attachment%3Bfilename=%22ForgeGradle-1.2-1.0.0-javadoc.jar%22&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHAqOi8vZDI5dnprNG93MDd3aTcuY2xvdWRmcm9udC5uZXQvZDEzM2QwYWI1N2YwZTFhOTk2ZmExMzY3ZWIzZTVjN2ViYmQxODYwYTVkNTA1YTY5NzkxZDc3NDY0NTdhY2IwYT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPWF0dGFjaG1lbnQlM0JmaWxlbmFtZSUzRCUyMkZvcmdlR3JhZGxlLTEuMi0xLjAuMC1qYXZhZG9jLmphciUyMiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTYxMjkzNzUzMH0sIklwQWRkcmVzcyI6eyJBV1M6U291cmNlSXAiOiIwLjAuMC4wLzAifX19XX0_&Signature=XwmQYLdJSkb~0fNr36ebj1vzn1KXmufRtaP9Mj9y9thRAtpAbyImJU-31VhtEhV56ijN2-tBtT5m0rNPSq-f6AEvngoEZURUUpE-vGVb1UFwiGnVnTuDKQJUvao3gZTj-fq9kRCLosJGYJiVbLG3GTXQKYoxL9fQh2ttwS3eFKAyFylcjyvsmSkEAekomfjB8OhUpgc~B6CE0Vg6aOm3ZBaKNAtfX4~cccnFrz4PB836WD68y8qX9jN5IQMCAiDy0881chyPIjGM7c30aycANw5c8StHSRy~gWo5WL7TGpq19X5q2pK-sT8Vq0I3AJJZ95XW9N3GWj0WR7IKqOQTAg__&Key-Pair-Id=APKAIFKFWOMXM2UMTSFA) invalid: 403 Forbidden. Text: "<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access denied</Message></Error>"
error url: https://d29vzk4ow07wi7.cloudfront.net/d133d0ab57f0e1a996fa1367eb3e5c7ebbd1860a5d505a69791d7746457acb0a?response-content-disposition=attachment%3Bfilename=%22ForgeGradle-1.2-1.0.0-javadoc.jar%22&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHAqOi8vZDI5dnprNG93MDd3aTcuY2xvdWRmcm9udC5uZXQvZDEzM2QwYWI1N2YwZTFhOTk2ZmExMzY3ZWIzZTVjN2ViYmQxODYwYTVkNTA1YTY5NzkxZDc3NDY0NTdhY2IwYT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPWF0dGFjaG1lbnQlM0JmaWxlbmFtZSUzRCUyMkZvcmdlR3JhZGxlLTEuMi0xLjAuMC1qYXZhZG9jLmphciUyMiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTYxMjkzNzUzMH0sIklwQWRkcmVzcyI6eyJBV1M6U291cmNlSXAiOiIwLjAuMC4wLzAifX19XX0_&Signature=XwmQYLdJSkb~0fNr36ebj1vzn1KXmufRtaP9Mj9y9thRAtpAbyImJU-31VhtEhV56ijN2-tBtT5m0rNPSq-f6AEvngoEZURUUpE-vGVb1UFwiGnVnTuDKQJUvao3gZTj-fq9kRCLosJGYJiVbLG3GTXQKYoxL9fQh2ttwS3eFKAyFylcjyvsmSkEAekomfjB8OhUpgc~B6CE0Vg6aOm3ZBaKNAtfX4~cccnFrz4PB836WD68y8qX9jN5IQMCAiDy0881chyPIjGM7c30aycANw5c8StHSRy~gWo5WL7TGpq19X5q2pK-sT8Vq0I3AJJZ95XW9N3GWj0WR7IKqOQTAg__&Key-Pair-Id=APKAIFKFWOMXM2UMTSFA
error url has %3D: false

NPE in ApacheRequestProducer when "http://" is requested

Digest authentition: using the realm of the 401 response to an initial incomplete request for an immediate second complete request including realm.

ResponseException is no longer serializable starting from 1.4.0 (breaking change)

https://youtrack.jetbrains.com/issue/KTOR-1552

Previously a ResponseException could be serialized. See: https://github.com/ktorio/ktor/issues/1256. The Transient annotation was removed in a recent commit for the HttpResponse field. Now an exception is thrown when serializing a ResponseException.

Stack trace:

io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
java.io.NotSerializableException: io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)
	at ExceptionsTest.serialize(ExceptionsTest.kt:26)
	at ExceptionsTest.testResponseExceptionSerializable(ExceptionsTest.kt:17)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)

Here is a test to reproduce the issue:

import io.ktor.client.call.*
import io.ktor.client.features.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.util.date.*
import io.ktor.utils.io.*
import java.io.*
import kotlin.coroutines.*
import kotlin.test.*

class ExceptionsTest {

    @Test
    fun testResponseExceptionSerializable() {
        val exception = createResponseException()

        val serialized = serialize(exception)
        val deserialized = deserialize(serialized)

        deserialized as ResponseException
    }

    private fun serialize(obj: Any): ByteArray {
        val b = ByteArrayOutputStream()
        val o = ObjectOutputStream(b)
        o.writeObject(obj)
        return b.toByteArray()
    }

    private fun deserialize(bytes: ByteArray): Any? {
        val b = ByteArrayInputStream(bytes)
        val o = ObjectInputStream(b)
        return o.readObject()
    }
}

private fun createResponseException(): ResponseException = ResponseException(object : HttpResponse() {
    override val call: HttpClientCall
        get() = TODO("Not yet implemented")
    override val status: HttpStatusCode
        get() = TODO("Not yet implemented")
    override val version: HttpProtocolVersion
        get() = TODO("Not yet implemented")
    override val requestTime: GMTDate
        get() = TODO("Not yet implemented")
    override val responseTime: GMTDate
        get() = TODO("Not yet implemented")
    override val content: ByteReadChannel
        get() = TODO("Not yet implemented")
    override val headers: Headers
        get() = TODO("Not yet implemented")
    override val coroutineContext: CoroutineContext
        get() = TODO("Not yet implemented")

    override fun toString(): String = "FakeCall"
}, cachedResponseText = "Fake text")

ClosedReceiveChannelException when making request with CIO engine using a proxy to https

https://youtrack.jetbrains.com/issue/KTOR-1458

version: 1.4.3

when use a http proxy, get the exception.

use no proxy, it's ok.

code


val httpClient = HttpClient(CIO) {
    engine {
        proxy = httpProxy
    }
}

val c : String = httpClient.get(url)

build.gradle.kts

// ...

dependencies {
    implementation("org.jetbrains.kotlinx", "kotlinx-serialization-json", "1.0.1")
    implementation("io.ktor", "ktor-client-core", "1.4.3")
    implementation("io.ktor", "ktor-client-cio", "1.4.3")
    implementation("io.ktor", "ktor-client-serialization-jvm", "1.4.3")
    implementation("org.jetbrains.kotlinx", "kotlinx-coroutines-core", "1.4.2")

    // test
}

exception:

Channel was closed
kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed
	at kotlinx.coroutines.channels.Closed.getReceiveException(AbstractChannel.kt:1107)
	at kotlinx.coroutines.channels.AbstractChannel$ReceiveElement.resumeReceiveClosed(AbstractChannel.kt:906)
	at kotlinx.coroutines.channels.AbstractSendChannel.helpClose(AbstractChannel.kt:322)
	at kotlinx.coroutines.channels.AbstractSendChannel.close(AbstractChannel.kt:251)
	at kotlinx.coroutines.channels.ChannelCoroutine.close(ChannelCoroutine.kt)
	at kotlinx.coroutines.channels.SendChannel$DefaultImpls.close$default(Channel.kt:105)
	at io.ktor.network.tls.TLSClientHandshake$input$1.invokeSuspend(TLSClientHandshake.kt:87)
	(Coroutine boundary)
	at kotlinx.coroutines.CompletableDeferredImpl.await(CompletableDeferred.kt:86)
	at io.ktor.client.engine.cio.Endpoint.execute(Endpoint.kt:76)
	at io.ktor.client.engine.cio.CIOEngine.execute(CIOEngine.kt:81)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
	(Coroutine creation stacktrace)
	at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:122)
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:30)
	at kotlinx.coroutines.BuildersKt.async$default(Unknown Source)
	at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:81)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:66)
	at io.ktor.client.engine.HttpClientEngine$install$1.invoke(HttpClientEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:128)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:92)
	at io.ktor.client.features.HttpSend$Feature$install$1.invoke(HttpSend.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:123)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:86)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invoke(HttpCallValidator.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:37)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invoke(HttpRequestLifecycle.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:188)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at myfunction(...)
	at LoginTest$testLogin$1.invokeSuspend(LoginTest.kt:18)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at mytest(...)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:101)
	at kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:101)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:129)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:86)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:37)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:188)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at myfunction(...)
	at mytest(...)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:37)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:188)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at myfunction(...)
	at mytest(...)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:188)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at myfunction(...)
	at mytest(...)
Caused by: kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed
	at kotlinx.coroutines.channels.Closed.getReceiveException(AbstractChannel.kt:1107)
	at kotlinx.coroutines.channels.AbstractChannel$ReceiveElement.resumeReceiveClosed(AbstractChannel.kt:906)
	at kotlinx.coroutines.channels.AbstractSendChannel.helpClose(AbstractChannel.kt:322)
	at kotlinx.coroutines.channels.AbstractSendChannel.close(AbstractChannel.kt:251)
	at kotlinx.coroutines.channels.ChannelCoroutine.close(ChannelCoroutine.kt)
	at kotlinx.coroutines.channels.SendChannel$DefaultImpls.close$default(Channel.kt:105)
	at io.ktor.network.tls.TLSClientHandshake$input$1.invokeSuspend(TLSClientHandshake.kt:87)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Client: NPE in FormDataContentKt -> Input.copyTo

https://youtrack.jetbrains.com/issue/KTOR-1349

I've received the following exception report for an Android app.
It was sporadic and I cannot reproduce the problem. The problem seems to lie within Ktor itself so I'll simply share what I can.

FATAL EXCEPTION: k.a0 Dispatcher = okhttp3.OkHttpClient Dispatcher
java.lang.NullPointerException
	at h.b.a.g.j.b$a.l(Unknown Source:58) = io.ktor.client.request.forms.FormDataContentKt$copyTo$2.invokeSuspend(java.lang.Object)
	at kotlin.c0.k.a.a.n(Unknown Source:9) = kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(java.lang.Object)
	at kotlinx.coroutines.a1.run(Unknown Source:93)
	at kotlinx.coroutines.d3.b.b0(Unknown Source:0)
	at kotlinx.coroutines.d3.b$a.c(Unknown Source:14)
	at kotlinx.coroutines.d3.b$a.m(Unknown Source:28)
	at kotlinx.coroutines.d3.b$a.run(Unknown Source:0)

This is the relevant mapping:

io.ktor.client.request.forms.FormDataContentKt$copyTo$2 -> h.b.a.g.j.b$a:
    io.ktor.utils.io.WriterSuspendSession p$ -> p
    java.lang.Object L$0 -> q
    io.ktor.utils.io.core.Input $this_copyTo -> s
    int label -> r
    java.lang.Object invoke(java.lang.Object,java.lang.Object) -> M
    kotlin.coroutines.Continuation create(java.lang.Object,kotlin.coroutines.Continuation) -> d
    java.lang.Object invokeSuspend(java.lang.Object) -> l

The relevant code from Ktor:
https://github.com/ktorio/ktor/blob/a6d68420ee0b272cd09d235057427a6601736487/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L154

private suspend fun Input.copyTo(channel: ByteWriteChannel) {
    if (this is ByteReadPacket) {
        channel.writePacket(this)
        return
    }

    channel.writeSuspendSession {
        while (!this@copyTo.endOfInput) {
            tryAwait(1)
            val buffer = request(1)!!                                   // <-- likely cause
            val size = this@copyTo.readAvailable(buffer)

            if (size < 0) continue
            written(size)
        }
    }
}

CIO: client engine exceptions are both logged and thrown

https://youtrack.jetbrains.com/issue/KTOR-1127

In the CIO client engine exceptions like SocketTimeoutException are logged to standard error by the coroutines' uncaught exception handler and thrown by client.get().

import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*

suspend fun main(): Unit = withContext(Dispatchers.Default) {
    HttpClient(CIO) {
        install(HttpTimeout) {
            socketTimeoutMillis = 5
        }
    }.use { client ->
        try {
            client.get<HttpResponse>("http://www.google.com").readText()
        } catch (e: Throwable) {
            println("Caught: $e")
        }
    }
}

Output:

Exception in thread "kotlinx.coroutines.DefaultExecutor" java.net.SocketTimeoutException
	at io.ktor.network.util.UtilsKt$withSocketTimeout$2.invokeSuspend(Utils.kt:23)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:32)
	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:113)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:188)
	at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:122)
	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
	at kotlinx.coroutines.CancellableContinuationImpl.cancel(CancellableContinuationImpl.kt:177)
	at kotlinx.coroutines.CancellableContinuationImpl.parentCancelled$kotlinx_coroutines_core(CancellableContinuationImpl.kt:184)
	at kotlinx.coroutines.ChildContinuation.invoke(JobSupport.kt:1484)
	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1511)
	at kotlinx.coroutines.JobSupport.tryMakeCancelling(JobSupport.kt:792)
	at kotlinx.coroutines.JobSupport.makeCancelling(JobSupport.kt:752)
	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:668)
	at kotlinx.coroutines.JobSupport.cancelCoroutine(JobSupport.kt:655)
	at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:128)
	at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:497)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
	at java.base/java.lang.Thread.run(Thread.java:832)

Caught: io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=http://www.google.com/, socket_timeout=5] ms

HttpRedirect feature alters Location header value

CIO Engine's HttpClient may fail when trying to send large size binary data.

https://youtrack.jetbrains.com/issue/KTOR-1247

Ktor Version and Engine Used (client or server and name)

Client CIO, 1.4.1

Describe the bug

HttpClient with CIO Engine sometimes sends an invalid multipart/form-data request when it includes large size binary data.
When it comes to invalid requests, it does not seem to contain a trailing multipart/form-data request boundary.
(There doesn't seem to be enough content in the request body.)

java.nio.BufferOverflowException seems to occur in the following process.
https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L123

This behavior is not reproduced in the case of HttpClient with Apache Engine.

To Reproduce

Sample code is below.

@KtorExperimentalAPI
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    routing {
        post("/multipart") {
            val multipartData = call.receiveMultipart()
            multipartData.forEachPart {
                if (it is PartData.FileItem) {
                    println(
                        withContext(Dispatchers.IO) {
                            it.streamProvider().readAllBytes().size
                        }
                    )
                }
            }
            call.respondText("test")
        }
        get("/execute/cio") {
            repeat(10) {
                request(HttpClient(CIO))
            }
            call.respondText("success")
        }
        get("/execute/apache") {
            repeat(10) {
                request(HttpClient(Apache))
            }
            call.respondText("success")
        }
    }
}

suspend fun request(client: HttpClient) {
    val body = formData {
        val headers = headersOf(
            HttpHeaders.ContentDisposition,
            ContentDisposition.File
                .withParameter(ContentDisposition.Parameters.Name, "file")
                .withParameter(ContentDisposition.Parameters.FileName, "test.jpg")
                .toString()
        )
        appendInput("file", headers) {
            // dummy 10MB binary file
            ByteArray(10_000_000).inputStream().asInput()
        }
    }
    client.use {
        it.submitFormWithBinaryData<Unit>(
            url = "http://localhost:8080/multipart",
            formData = body
        )
    }
}

(full code is in https://github.com/ma2Cta/ktor-multipart-request-test/blob/main/src/Application.kt )

Start the server.
Run $ curl http://localhost:8080/execute/apache a few times.

10 times 10000000 will be printed as expected.

Run $ curl http://localhost:8080/execute/cio a few times.

You won't get 10000000 10 times, and you'll probably get a number less than 10000000 the first few times

Expected

The correct request is sent even in the case of CIO Engine.

Timeout feature: android engine throws Java's SocketTimeoutException instead of ConnectTimeoutException

Client sometimes doesn't do response validation

Make `body` nullable for request builder

Breaking change: formUrlEncode no longer encodes = in the parameters' values

ktor-client-js - Cannot read property Json of undefined

iOS client fails with CoroutinesInternalError when Logging is used

https://youtrack.jetbrains.com/issue/KTOR-924

Creating a simple HttpClient with the Logging feature included fails with the attached stack trace. The failure occurs both for real web calls in-app, as well as for a unit test using MockEngine. Both the test and the in-app call succeed if the logging configuration is removed.

I have a simple reproducer project attached, and also available at https://github.com/russhwolf/Ktor14

Note that my test configuration is using version 1.3.9-native-mt of kotlinx.coroutines, because version 1.3.9 leads to KTOR-915 / KTOR-919

kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[EventLoopImpl@36817bc8, Continuation @ $split$lambda-0$<anonymous>_3COROUTINE$2]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlin.Error#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
	at kotlinx.coroutines.CoroutinesInternalError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Exceptions.common.kt:28)
	at kotlinx.coroutines.DispatchedTask#handleFatalException(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:64)
	at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
	at kotlinx.coroutines#runEventLoop(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:80)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67)
	at kotlinx.coroutines#runBlocking(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50)
	at com.example#runBlocking(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:8)
	at com.example#runBlocking$default(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:7)
	at com.example.ApiClientTest#mockWebCall(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:20)
	at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.invoke#internal(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
	at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.$<bridge-UNNN>invoke(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
	at kotlin.native.internal.test.BaseClassSuite.TestCase#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestSuite.kt:85)
	at kotlin.native.internal.test.TestRunner.run#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:236)
	at kotlin.native.internal.test.TestRunner.runIteration#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:258)
	at kotlin.native.internal.test.TestRunner#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:273)
	at kotlin.native.internal.test#testLauncherEntryPoint(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:22)
	at kotlin.native.internal.test#main(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:26)
	at <global>.Konan_start(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:25)
	at <global>.Init_and_run_start(Unknown Source)
	at <global>.start(Unknown Source)
	at kotlin.Exception#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:25)
	at kotlin.RuntimeException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:36)
	at kotlinx.coroutines.CompletionHandlerException#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Exceptions.common.kt:13)
	at kotlinx.coroutines.JobSupport.notifyCompletion#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:365)
	at kotlinx.coroutines.JobSupport.completeStateFinalization#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:326)
	at kotlinx.coroutines.JobSupport.finalizeFinishingState#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:241)
	at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:922)
	at kotlinx.coroutines.JobSupport.tryMakeCompleting#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:875)
	at kotlinx.coroutines.JobSupport#makeCompletingOnce(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:840)
	at kotlinx.coroutines.AbstractCoroutine#resumeWith(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:111)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43)
	at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:53)
	at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
	at kotlin.Throwable#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23)
	at kotlin.Exception#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23)
	at kotlin.RuntimeException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34)
	at kotlin.native.concurrent.InvalidMutabilityException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:22)
	at <global>.ThrowInvalidMutabilityException(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:92)
	at <global>.MutationCheck(Unknown Source)
	at kotlinx.coroutines.AwaitAll.AwaitAllNode.<set-handle>#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:94)
	at kotlinx.coroutines.AwaitAll.await#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:68)
	at kotlinx.coroutines.$awaitAllCOROUTINE$1#invokeSuspend(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:38)
	at kotlinx.coroutines#awaitAll@kotlin.collections.Collection<kotlinx.coroutines.Deferred<0:0>>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:37)
	at io.ktor.util.$split$lambda-0COROUTINE$3.invokeSuspend#internal(/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/ByteChannels.kt:29)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:30)
	at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:39)
	at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
	at kotlinx.coroutines#runEventLoop(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:80)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67)
	at kotlinx.coroutines#runBlocking(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50)
	at com.example#runBlocking(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:8)
	at com.example#runBlocking$default(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:7)
	at com.example.ApiClientTest#mockWebCall(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:20)
	at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.invoke#internal(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
	at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.$<bridge-UNNN>invoke(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
	at kotlin.native.internal.test.BaseClassSuite.TestCase#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestSuite.kt:85)
	at kotlin.native.internal.test.TestRunner.run#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:236)
	at kotlin.native.internal.test.TestRunner.runIteration#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:258)
	at kotlin.native.internal.test.TestRunner#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:273)
	at kotlin.native.internal.test#testLauncherEntryPoint(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:22)
	at kotlin.native.internal.test#main(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:26)
	at <global>.Konan_start(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:25)
	at <global>.Init_and_run_start(Unknown Source)

ktor-client-apache: thread stuck in ByteBufferChannel.readRemainingSuspend

https://youtrack.jetbrains.com/issue/KTOR-1463

In production I've got two threads stuck in 100% CPU usage due to Ktor.

The server is running without debugging agent, so I cannot debug the exact scenario just yet. I'll restart it today with debugging agent running so that I can investigate further the next time the issue occurs.

When creating thread dumps I get one of the following two outputs. Note the high CPU usage and that the threads seem to be stuck in ByteBufferChannel.readRemainingSuspend.

"DefaultDispatcher-worker-2" #423 daemon prio=5 os_prio=0 cpu=51203112.18ms elapsed=53179.31s tid=0x00007f6fc0052510 nid=0xa9b2 runnable  [0x00007f6f0ebe9000]
   java.lang.Thread.State: RUNNABLE
	at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible$default(ByteBufferChannel.kt:2678)
	at io.ktor.utils.io.ByteBufferChannel.readRemainingSuspend(ByteBufferChannel.kt:2231)
	at io.ktor.utils.io.ByteBufferChannel$readRemainingSuspend$1.invokeSuspend(ByteBufferChannel.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

"DefaultDispatcher-worker-2" #423 daemon prio=5 os_prio=0 cpu=51253887.85ms elapsed=53230.30s tid=0x00007f6fc0052510 nid=0xa9b2 runnable  [0x00007f6f0ebe9000]
   java.lang.Thread.State: RUNNABLE
	at io.ktor.utils.io.core.internal.UnsafeKt.prepareWriteHead(Unsafe.kt:160)
	at io.ktor.utils.io.ByteBufferChannel.readRemainingSuspend(ByteBufferChannel.kt:3060)
	at io.ktor.utils.io.ByteBufferChannel$readRemainingSuspend$1.invokeSuspend(ByteBufferChannel.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Core

Significant performance penalty with socket write() and file read() on JVM

ByteBufferChannel.readRemaining suspend when should not

Fix socket self connect problem

https://youtrack.jetbrains.com/issue/KTOR-1465

I already made a pr on github: https://github.com/ktorio/ktor/pull/2237

Subsystem
ktor-network-jvm

Motivation
TCP has a well known self-connect problem, which client can connect to the client itself without any program listen on the port.

Reproduce Code :

fun testSelfConnect() {
    runBlocking {
        // Find a port that would be used as a local address.
        val port = getAvailablePort()

        val tcpSocketBuilder = aSocket(ActorSelectorManager(Dispatchers.IO)).tcp()
        // Try to connect to that address repeatedly.
        for (i in 0 until 100000) {
            try {
                val socket = tcpSocketBuilder.connect(InetSocketAddress("127.0.0.1", port))
                println("connect to self succeed: ${socket.localAddress} to ${socket.remoteAddress}")
                System.`in`.read()
                break
            } catch (ex: Exception) {
                // ignore
            }
        }
    }
}

// since new linux kernel version introduce a feature, new bind port number will always be odd number
// and connect port will always be even, so we find a random even port with while loop
// see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=07f4c90062f8fc7c8c26f8f95324cbe8fa3145a5
private fun getAvailablePort(): Int {
    while (true) {
        val port = ServerSocket().apply {
            bind(InetSocketAddress("127.0.0.1", 0))
            close()
        }.localPort

        if (port % 2 == 0) {
            return port
        }

        try {
            // try bind the next even port
            ServerSocket().apply {
                bind(InetSocketAddress("127.0.0.1", port + 1))
                close()
            }
            return port + 1
        } catch (ex: Exception) {
            // ignore
        }
    }
}

Running the above code whill result in a tcp connection with src addr equal to dst addr and src port equal to dst port.

If this happens, client will behave abnormal because it connect to the wrong server，server will bind fail on this machine.

Other programming language also have this problem, eg golang: https://golang.org/issue/2690

Fix this problem is easy, just check if the local and remote socket pair are equal.

Incorrect encoding function used for URL path by URLBuilder

Input.readTextExactBytes(n) on empty input different behavior per platform

ktor-io: JVM shared function decrease performance starting from 1.4.0

Experimental API and compatibility guarantees

https://youtrack.jetbrains.com/issue/KTOR-1035

Problem

Currently, we have a public commitment to not break any API and ABI. This is generally good.
The problem is that it makes it almost impossible for a new experimental feature/API to grow and entirely stops evolution since major versions are released too slowly.

Proposed solution

The idea is to relax guarantees for all API that is marked with experimental annotations similar to Kotlin.

Rules

in a patch version: no changes allowed, deprecations with level WARNING could be introduced for existing API.
in a minor version: both source and binary changes allowed except for complete removal. All changes should include migration/deprecation. Hidden deprecated APIs should be introduced for binary compatibility.
in a major version: any changes including complete removal but only through the deprecation cycle.

Requirements for experimental API:

experimental API should be marked with an experimental annotation requiring opt-in
an experimental annotation should have level ERROR and should have a reasonable and clear description
for a single or group of new experimental functions/classes/APIs having the same purpose and/or related functionality, it is required to introduce a new experimental annotation

Note: the consequence is that you can't use KtorExperimentalAPI annotation for new API. So one should always create new experimental annotations for now.
The other consequence is that we should revisit all KtorExperimentalAPI usages and either stabilize them or migrate to new experimental annotations.

Requirements for experimental API annotation:

An experimental annotation class should:

require opt-in with level ERROR (@RequiresOptIn(level = ERROR));
have a reasonable and clear description;
have a reference to a ktor ticket that introduced this API and/or describes why the API is experimental and that is useful for tracking the concrete experimental feature status;
have retention BINARY, unless there is a reason to have RUNTIME;
be documented;
contain a disclamer in the kdoc about the risk of using experimental features.

Note: RUNTIME retention is not recommended since the ability to analyze it in runtime makes it more difficult to remove the annotation due to indirect and unobvious references. Retention SOURCE is not acceptable because it is how @RequiresOptIn works in Kotlin.

Experimental API Deprecation cycle

A deprecation cycle for an API that is intended to be removed or changed consist of the following steps:

In any release, make the API deprecated with level WARNING by applying one of the following:

@Deprecated annotation with level WARNING, provide a clear message describing what will be changed and what exact migration is required, provide a "replace with"
a new annotation requiring opt-in with level WARNING and the description

In the next minor release, promote the deprecation level from WARNING to ERROR
In the next minor release after step 2, change the deprecation level from ERROR to HIDDEN, keep the implementation working despite that it is hidden (to keep binary compatibility)
In the next major release, remove the old API

Please note that using an opt-in annotation for deprecation doesn't provide the ability to write a "replace with". Thus, it only makes sense to use it for removing old API. A good example is kotlinx.coroutines ObsoleteCoroutinesApi

If some of the rules are impossible to comply (for example, it is impossible to provide a "replace with" action or impossible to keep a hidden function working), an author of change should initiate a discussion at least with PR reviewer. When decided, it could be also discussed in the team or even publically for a quite popular experimental API. If it was decided to skip some of the requirements, it is important to document that decision and reasons in the PR.

KtorExperimentalAPI deprecation

@KtorExperimentalAPI annotation is going to be deprecated in the following stages:

Revisit all existing API marked with the annotation and stabilize when possible without breaking changes in a minor version(s). Mark the annotation deprecated with level WARNING.
In major version (2.0.0), revisit again. When impossible to stabilize, introduce new experimental annotations and use them instead. Ensure no KtorExperimentalAPI usages. Promote its deprecation to level ERROR.
In the next major version (3.0.0) deprecate it with level HIDDEN. Use text search to ensure no text occurrences.
In the next major version (4.0.0) remove it.

The reason why we deprecate it is that it introduces a way of "anonymous" experiment. An experimental API should be introduced with a clear reason and plan. A plan should cover only one particular feature/functionality. An experimental feature should have a more or less clear lifecycle.

The other disadvantage of this annotation is that there is no easy way to de-opt-in users from it. Also, it invites most brave users to opt-in globally for the whole project (just as we did for ExperimentalCoroutinesApi). This is very bad because in this case experimental usages will become uncontrolled and the user doesn't realize anymore what danger it is. So to prevent such problems, we should stop using it and migrate to a better approach.

Example

Here is an example of an experimental API:

/**
 * API marked with this annotation is a part of experimental Z feature. 
 * The feature preview provides the ability to work with Z. 
 * Please see KTOR-number for details and stabilization plan.
 * 
 * This is an API preview and it's not recommended for production development 
 * since it could be changed or removed in the future and 
 * not guaranteed to remain functional and its behaviour may also change in any non-patch release.
 */
@RequiresOptIn("Z feature is experimental and not recommended for production development.", level = ERROR)
@Retention(BINARY)
annotation class ZFeature

/**
 * Z feature's context
 */
@ZFeature
class ZContext

/** 
 * Configures Z feature and invokes [block] with running instance.
 */
@ZFeature
fun withZ(block: () -> Unit) = ...

Docs

java.net.MalformedURLException: unknown protocol: ws KTOR android

https://youtrack.jetbrains.com/issue/KTOR-1410

as stated in documentation I want to create websocket in Server and connect to it from Android my code for Android:

     class MainActivity : AppCompatActivity() {
  override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
val client = HttpClient {
    install(WebSockets)
}
lifecycleScope.launchWhenCreated {
    withContext(Dispatchers.IO) {
        client.ws(
                method = HttpMethod.Get,
                host = "ws://192.168.43.4",
                port = 23569, path = ""
        ) { // this: DefaultClientWebSocketSession

            // Send text frame.
            send("Hello, Text frame")

            // Send text frame.
            send(Frame.Text("Hello World"))

         

            // Receive frame.
            val frame = incoming.receive()
            when (frame) {
                is Frame.Text -> println(frame.readText())
                is Frame.Binary -> println(frame.readBytes())
            }
        }
    }
}

}
}

but it gives the following error:

 java.net.MalformedURLException: unknown protocol: ws
    at java.net.URL.<init>(URL.java:597)
    at java.net.URL.<init>(URL.java:487)
    at java.net.URL.<init>(URL.java:436)
    at io.ktor.client.engine.android.AndroidClientEngine.getProxyAwareConnection(AndroidClientEngine.kt:102)
    at io.ktor.client.engine.android.AndroidClientEngine.execute(AndroidClientEngine.kt:47)
    at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

https://stackoverflow.com/questions/65113603/java-net-malformedurlexception-unknown-protocol-ws-ktor-android

OAuth authentication documentation is confusing because of incomplete code example

Full example for configuring response validation is missing

Duplicate entry "Features" in Server docs

Missing dependency information in the Timeout topic

Web feedback from "https://ktor.io/docs/nopage.html"

Guides topic is empty

No content for the Metrics feature topic

Apache client engine is documented to support HTTP/2 but doesn't

Server features list is empty

Missing dependency information the Client Auth topic

Add information about required artifacts to the WebSockets topic

Outdated dependency information in the Client Logging topic

Center the docs

Generator

Ktor generator fails to generate when Auth-Oauth is selected

DoubleReceive and Kotlinx.serialization features are missing in generator

Infrastructure

settings.gradle: if (native_targets_enabled) does nothing

Add build parameter to build ktor with JVM IR compiler

CI does not trigger build on PR to samples repo

Invalid publication 'kotlinMultiplatform' for task :publishKotlinMultiplatformPublicationToMavenLocal on master

IntelliJ IDEA Plugin

Ktor plugin - A new project with Digest authentication uses deprecated API

Server

Memory leak on Https request

Unexpected exception when using Session feature: "Using blocking primitives on this dispatcher is not allowed"

https://youtrack.jetbrains.com/issue/KTOR-1452

kotlin: 1.4.20
kotlinx.coroutines: 1.4.2

Sessions feature is configured in a rather standard manner:

        install(Sessions) {
            cookie<UserIdPrincipal>(
                Cookies.SESSION_ID,
                storage = directorySessionStorage(sessionStorageRootDir)
            ) {
                cookie.path = "/"
                cookie.httpOnly = true
                cookie.extensions["SameSite"] = "Lax"
            }
        }

When it is called during a pipeline execution, an unexpected exception is thrown.

java.lang.IllegalStateException: Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
	at io.ktor.utils.io.jvm.javaio.BlockingKt.ensureParkingAllowed(Blocking.kt:302) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.access$ensureParkingAllowed(Blocking.kt:1) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.InputAdapter.<init>(Blocking.kt:30) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream$default(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.DirectoryStorage.read(DirectoryStorage.kt:46) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invokeSuspend(Cache.kt:82) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invoke(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2$1.invokeSuspend(Cache.kt:38) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	(Coroutine boundary) ~[?:?]
	at kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:101) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.sessions.ReferenceCache.getOrCompute$suspendImpl(Cache.kt:86) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage.read(CacheStorage.kt:20) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionTrackerById.load(SessionTrackerById.kt:34) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:215) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:259) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:110) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:102) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:101) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:142) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118) ~[ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:118) ~[ktor-server-netty-jvm-1.4.3.jar:1.4.3]
Caused by: java.lang.IllegalStateException: Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
	at io.ktor.utils.io.jvm.javaio.BlockingKt.ensureParkingAllowed(Blocking.kt:302) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.access$ensureParkingAllowed(Blocking.kt:1) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.InputAdapter.<init>(Blocking.kt:30) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream$default(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.DirectoryStorage.read(DirectoryStorage.kt:46) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invokeSuspend(Cache.kt:82) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invoke(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2$1.invokeSuspend(Cache.kt:38) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.4.20.jar:1.4.20-release-308 (1.4.20)]
	at kotlinx.coroutines.internal.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:342) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:30) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable$default(Cancellable.kt:27) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt:91) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.async(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async$default(Builders.common.kt:84) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.async$default(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.sessions.BaseCache$getOrCompute$2.apply(Cache.kt:37) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2.apply(Cache.kt:31) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_251]
	at io.ktor.sessions.BaseCache.getOrCompute(Cache.kt:36) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache.getOrCompute$suspendImpl(Cache.kt:86) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache.getOrCompute(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseTimeoutCache.getOrCompute(Cache.kt:163) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage.read(CacheStorage.kt:20) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionTrackerById.load(SessionTrackerById.kt:34) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:215) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:63) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:110) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:102) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:194) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:101) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:142) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118) [ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt) [ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:118) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:43) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:34) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:216) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251]

Server losing channel exceptions at receive

Jackson: Crash when sending large responses in 1.4.2

https://youtrack.jetbrains.com/issue/KTOR-1369

Hi there,

We've discovered that Ktor 1.4.2 appears to crash out when sending large responses, using the Jackson feature. I think it might be related to KTOR-1342, but it's also happening on sending. Downgrading to ktor-jackson:1.4.1 seems to fix the issue.

The error logged is:

Exception in thread "eventLoopGroupProxy-3-1" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@2b5e16e4, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@114e04cb]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
        at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2258)
        at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:929)
        at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1953)
        at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:305)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
        at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
        at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)

I've created a sample project for you to try - https://github.com/SkyeWelch/ktor-1342-error-sample . You can run it with ./gradlew run, and if it's the same as on my machine see the following responses (I'm using httpie to make requests):

→ http localhost:8080/small
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
transfer-encoding: chunked

{
    "string": "small string"
}

→ http localhost:8080/large
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
transfer-encoding: chunked


http: error: ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

Please let me know if you need anything else.

Thanks,
Skye

GMTDate: Operator overload (plus/minus) for kotlin.time.Duration

Request headers exceeding expected threshold are not handled correctly

Upgrade Netty to 4.1.54.Final

"Wrong HEX escape": gracefully handle invalid URLs

https://youtrack.jetbrains.com/issue/KTOR-1308

We recently got some alerts in our webserver because ktor threw an uncaught exception. Ideally these are handled and a 400 is sent back.

Here is the exception:
Wrong HEX escape: %uf, in /%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/winnt/win.ini

Stacktrace:

at io.ktor.http.CodecsKt.decodeImpl (Codecs.kt:197)
at io.ktor.http.CodecsKt.decodeScan (Codecs.kt:146)
at io.ktor.http.CodecsKt.decodeURLPart (Codecs.kt:140)
at io.ktor.http.CodecsKt.decodeURLPart$default (Codecs.kt:139)
at io.ktor.routing.RoutingResolveContext.parse (RoutingResolve.kt:70)
at io.ktor.routing.RoutingResolveContext.<init> (RoutingResolve.kt:49)
at io.ktor.routing.Routing.interceptor (Routing.kt:31)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend (Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke (Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend (ContentNegotiation.kt:107)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke (ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend (DefaultEnginePipeline.kt:120)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke (DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend (NettyApplicationCallHandler.kt:40)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke (NettyApplicationCallHandler.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched (Undispatched.kt:55)
at kotlinx.coroutines.CoroutineStart.invoke (CoroutineStart.kt:111)
at kotlinx.coroutines.AbstractCoroutine.start (AbstractCoroutine.kt:158)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch (Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch (Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest (NettyApplicationCallHandler.kt:30)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead (NettyApplicationCallHandler.kt:24)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600 (AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run (AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)

Netty HTTP/2 HEAD response hangs

Sessions + SSL (Netty)

https://youtrack.jetbrains.com/issue/KTOR-1292

edit: https://github.com/dm6801/ktor_sessions_ssl

when trying to use Session feature with cookies

    install(Sessions) {
        cookie<MySession>("MY_SESSION")
    }

accessing https:/localhost:443/ throws:

java.lang.UnsupportedOperationException: null
	(Coroutine boundary)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:55)
	(Coroutine creation stacktrace)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:188)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:38)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:821)
Caused by: java.lang.UnsupportedOperationException: null
	at io.ktor.server.netty.http2.NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64)
	at io.ktor.sessions.SessionTransportCookie.receive(SessionTransportCookie.kt:28)
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:204)
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:59)
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)

relevant line NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64):

override val cookies: RequestCookies
    get() = throw UnsupportedOperationException()

http://localhost:8080/ - works

Support for precompresed files

Digest authentication: cannot successfully pass authentication using curl or web browser

https://youtrack.jetbrains.com/issue/KTOR-1466

To reproduce run the server using the following code snippet:

fun main() {
    embeddedServer(Netty, port = 8081) {
        install(Authentication) {
            digest("digest") {
                val password = "123"
                realm = "testrealm@host.com"
                digestProvider { name, realm ->
                    "$name:$realm:$password".toByteArray()
                }
            }
        }
        routing {
            authenticate("digest") {
                get("/digest") {
                    call.respondText("ok")
                }
            }
        }
    }.start(wait = true)
}

Send the following HTTP request using curl:

curl -v  'http://localhost:8081/digest' --digest -u guest:123

I expect successfully pass authentication and get a response with a status code 200 but instead, I get the following response:

*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8081 (#0)
* Server auth using Digest with user 'guest'
> GET /digest HTTP/1.1
> Host: localhost:8081
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Digest realm="testrealm@host.com", nonce="43f48979ca35e2a5", algorithm="MD5"
< Content-Length: 0
< 
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8081/digest'
* Found bundle for host localhost: 0x7fbe62518750 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 8081 (#0)
* Server auth using Digest with user 'guest'
> GET /digest HTTP/1.1
> Host: localhost:8081
> Authorization: Digest username="guest", realm="testrealm@host.com", nonce="43f48979ca35e2a5", uri="/digest", response="ee8c56f444996c77cd1e5f7570035372", algorithm="MD5"
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
* Authentication problem. Ignoring this.
< WWW-Authenticate: Digest realm="testrealm@host.com", nonce="76d13a440ec99b1c", algorithm="MD5"
< Content-Length: 0
< 
* Connection #0 to host localhost left intact
* Closing connection 0

HTML Page response size above 4099 bytes causes fatal sever exceptions

https://youtrack.jetbrains.com/issue/KTOR-1462

I have an extremely weird issue with Ktor. The following exception is thrown after about 2-3 reloads of a page.

Exception in thread "eventLoopGroupProxy-3-2" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@259245dc, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@6200633a]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
	at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2262)
	at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:909)
	at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1952)
	at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:307)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:216)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	... 22 more

However, I can reliably avoid the issue when I remove some of the HTML code that is being sent back for this request.

So basically; when I change a portion of my HTML output for a route to be shorter than it should be, the error will never occur.

As soon as I put back the HTML, basically crossing some limit, the Exception is thrown always after the third/fourth reload.

I have created the following repository that demonstrates the issue (entirey reproducible for me): https://github.com/ApoY2k/exception-demo

I also included two HAR files, one with a working request/response and one with a failing one (notice that the reponse does work, but the exception is still thrown which causes other server issues down the line)

For future reference, the main class code is this: (also attached is a package of the whole repository)

package apoy2k.exception.demo

import io.ktor.application.*
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.*
import io.ktor.http.*
import io.ktor.html.*
import kotlinx.html.*

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)

@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    routing {
        get("/") {
            call.respondHtml {
                // 4058 works
                // 4059 throws exception
                for (x in 1..4058) {
                    +"a"
                }
            }
        }
    }
}

It seems that as soon as the response HTML is larger than 4099 bytes it crashes. Anything until that limit runs perfectly fine.

What is going on here? How can I possibly fix this?

Jackson: Crash when sending large responses in 1.4.3

https://youtrack.jetbrains.com/issue/KTOR-1409

Hi,

It seems that this issue didnt fix on ktor 1.4.2 (see KTOR-1369) and fired similar exception on Ktor 1.4.3 using the Jackson feature. Downgrading to ktor-jackson:1.4.1 seems to fix the issue.

The error logged is:

Exception in thread "eventLoopGroupProxy-3-1 @response-pipeline#2" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@17d44fe5, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@335313af]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:144)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:115)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
	at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2262)
	at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:909)
	at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1952)
	at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:307)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:216)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.internal.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.internal.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:104)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.internal.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.internal.DispatchedContinuation are in unnamed module of loader 'app')

	... 22 more

Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.

Parsing Authorization header throws IllegalArgumentException instead of returning null

Other

Implement Support for Compression Extensions for WebSocket (RFC 7692)

Upgrade gradle to 6.7.1 and enable fs watcher

testWriteXRaceCondition sometimes hangs on CI

Remove copyTo usage from ServerPipeline

Fix failing tests in master after 1.5.0 merge

Handle failure in reading request body

`ByteBufferChannel.readRemaining` doesn't read whole channel

Client: URL encode / escaping is wrong

The first call.receiveText() hangs until timeout

Fix configuration if project without VPN and cache

Update kotlin to 1.4.20

Wrong pool is used to release `IOBuffer` after `ByteChannelSequential.copyTo` from static initialized instance.

How to create a response interceptor that reads the current response from server and changes it conditionally

Ktor post(any - request) with execute

Not able to build iOS framework with Xcode 11.3.1 and ktor version 1.4.2 onwards. However, it works with ktor version 1.4.1

1.4.2

released 10th November 2020

Client

Add WebSocket support for the iOS

No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class kotlin.collections.List

https://youtrack.jetbrains.com/issue/KTOR-1343

I`m using KTOR for create Http requests in KMM Project (android + ios)
My requests don't work at all

Occur exception: No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class kotlin.collections.List

Configuration of my project

val serializationVersion = "1.0.0-RC"
val ktorVersion = "1.4.2"
val coroutinesVersion = "1.3.9-native-mt"
val sqlDelightVersion: String by project
val kotlinVersion = "1.4.0"
val koinVersion = "3.0.0-alpha-4"

sourceSets {
        val commonMain by getting {
            dependencies {
                implementation("org.jetbrains.kotlin:kotlin-reflect:$kotlinVersion")
                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion")
                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core-common:1.1.0")
                implementation("org.jetbrains.kotlinx:kotlinx-serialization-core:1.0.0-RC")

                implementation("io.ktor:ktor-client-cio:$ktorVersion")
                implementation("io.ktor:ktor-client-core:$ktorVersion")
                implementation("io.ktor:ktor-client-json:$ktorVersion")
                implementation("io.ktor:ktor-client-logging:$ktorVersion")
                implementation("io.ktor:ktor-client-serialization:$ktorVersion")

                implementation("com.squareup.sqldelight:runtime:$sqlDelightVersion")
                implementation("com.squareup.sqldelight:coroutines-extensions:$sqlDelightVersion")

                implementation("org.koin:koin-core:$koinVersion")
            }
        }

        val commonTest by getting {
            dependencies {
                implementation(kotlin("test-common"))
                implementation(kotlin("test-annotations-common"))
            }
        }

        val androidMain by getting {
            dependencies {
                implementation("io.ktor:ktor-client-android:$ktorVersion")
                implementation("io.ktor:ktor-client-core:$ktorVersion")
                implementation("io.ktor:ktor-client-json:$ktorVersion")
                implementation("io.ktor:ktor-client-logging:$ktorVersion")
                implementation("io.ktor:ktor-client-serialization:$ktorVersion")
                implementation("io.ktor:ktor-client-gson:$ktorVersion")
                implementation("io.ktor:ktor-client-okhttp:$ktorVersion")

                implementation("com.squareup.sqldelight:android-driver:$sqlDelightVersion")
                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.9")
            }
        }

        val androidTest by getting {
            dependencies {
                implementation(kotlin("test-junit"))
                implementation("junit:junit:4.13")
            }
        }

        val iosMain by getting {
            dependencies {
                implementation("io.ktor:ktor-client-ios:$ktorVersion")
                implementation("com.squareup.sqldelight:native-driver:$sqlDelightVersion")
            }
        }

My Code

import io.ktor.client.*
import io.ktor.client.engine.okhttp.*
import io.ktor.client.features.*
import io.ktor.client.features.json.*
import io.ktor.client.features.json.serializer.*
import io.ktor.client.features.logging.*
import io.ktor.util.*

actual class Network actual constructor() {

    @KtorExperimentalAPI
    actual fun getHttpClient(): HttpClient {
        return HttpClient(OkHttp) {
            defaultRequest {
                install(HttpTimeout) {
                    requestTimeoutMillis = 5000
                    connectTimeoutMillis = 3000
                }

                install(JsonFeature) {
                    serializer = KotlinxSerializer()
                }


                install(Logging) {
                    logger = Logger.ANDROID
                    level = LogLevel.ALL
                }
            }
        }
    }
}

My Object

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

@Serializable
data class OnboardingResponse(
    @SerialName("title") val title: String,
    @SerialName("subtitle") val subtitle: String,
    @SerialName("description") val description: String,
    @SerialName("image_url") val imageUrl: String
)

Http request

class OnboardingDataSource(
    private val httpClient: HttpClient
) : IOnboardingDataSource {

    private val mapper by lazy { OnboardingMapper() }

    override suspend fun getList(): List<OnboardingData> {
        return httpClient
            .get<List<OnboardingResponse>>(
                UrlProvider().getHostUrl().plus(UrlProvider.onboarding)
            )
            .map { mapper.transform(it) }
    }
}

"IllegalStateException: Cannot execute task because event loop was shut down" for native client

https://youtrack.jetbrains.com/issue/KTOR-1071

To reproduce run ./gradlew runDebugExecutableMacosX64 in the root directory of the attached project.
As a result, I get the following unexpected exception:

Uncaught Kotlin exception: kotlin.IllegalStateException: Cannot execute task because event loop was shut down
    at 0   loop_shutdown_coroutines.kexe       0x000000010b88270d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
    at 1   loop_shutdown_coroutines.kexe       0x000000010b87ba7b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
    at 2   loop_shutdown_coroutines.kexe       0x000000010b87bc3b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
    at 3   loop_shutdown_coroutines.kexe       0x000000010b87c14b kfun:kotlin.IllegalStateException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:70:44)
    at 4   loop_shutdown_coroutines.kexe       0x000000010ba5639e kfun:kotlinx.coroutines#loopWasShutDown(){}kotlin.Nothing + 222 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/CoroutineContext.kt:25:50)
    at 5   loop_shutdown_coroutines.kexe       0x000000010ba5887d kfun:kotlinx.coroutines.EventLoopImplPlatform#reschedule(kotlin.Long;kotlinx.coroutines.EventLoopImplBase.DelayedTask){} + 61 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:22:9)
    at 6   loop_shutdown_coroutines.kexe       0x000000010b9e657b kfun:kotlinx.coroutines.EventLoopImplBase.rescheduleAllDelayed#internal + 587 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:397:13)
    at 7   loop_shutdown_coroutines.kexe       0x000000010b9e2dde kfun:kotlinx.coroutines.EventLoopImplBase#shutdown(){} + 270 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:226:9)
    at 8   loop_shutdown_coroutines.kexe       0x000000010ba58d1c kfun:kotlinx.coroutines.EventLoopImpl#shutdown(){} + 140 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:34:15)
    at 9   loop_shutdown_coroutines.kexe       0x000000010b9e12a4 kfun:kotlinx.coroutines.EventLoop#decrementUseCount(kotlin.Boolean){} + 324 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:114:13)
    at 10  loop_shutdown_coroutines.kexe       0x000000010b9e137c kfun:kotlinx.coroutines.EventLoop#decrementUseCount$default(kotlin.Boolean;kotlin.Int){} + 124 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:108:5)
    at 11  loop_shutdown_coroutines.kexe       0x000000010ba5383b kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 1115 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:85:20)
    at 12  loop_shutdown_coroutines.kexe       0x000000010ba551f0 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 464 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67:9)
    at 13  loop_shutdown_coroutines.kexe       0x000000010ba52dcf kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 + 1311 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50:22)
    at 14  loop_shutdown_coroutines.kexe       0x000000010ba5330c kfun:kotlinx.coroutines#runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 + 348 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:33:15)
    at 15  loop_shutdown_coroutines.kexe       0x000000010b82bc48 kfun:#main(){} + 168 (/Users/Aleksei.Tirman/projects/jb/loop_shutdown_coroutines/src/macosX64Main/kotlin/App.kt:6:14)
    at 16  loop_shutdown_coroutines.kexe       0x000000010b82c5c0 Konan_start + 144 (/Users/Aleksei.Tirman/projects/jb/loop_shutdown_coroutines/src/macosX64Main/kotlin/App.kt:6:1)
    at 17  loop_shutdown_coroutines.kexe       0x000000010b831f4b Init_and_run_start + 107
    at 18  libdyld.dylib                       0x00007fff6a1c9cc9 start + 1
Unfinished workers detected, 1 workers leaked!
Use `Platform.isMemoryLeakCheckerActive = false` to avoid this check.
Abort trap: 6

Here is the similar issue.
I've reproduced this problem for both macOS and Linux targets.
Kotlin multiplatform: 1.4.10
Kotlin coroutines: 1.3.9-native-mt-2
Ktor: 1.4.0

Ktor WebSocket with OkHttpClient authenticator

https://youtrack.jetbrains.com/issue/KTOR-1397

I have that client


    private val client: HttpClient = HttpClient(OkHttp) {
        engine {
            preconfigured = okHttpClient
        }
    }.config {
        install(WebSockets)
        install(JsonFeature) { serializer = GsonSerializer() }
    }

okHttpClient :

    private fun okHttpClient(
        loggingInterceptor: Interceptor,
        tokenAuthenticator: Authenticator,
        pingIntervalSec: Long = 20
    ) = with(OkHttpClient.Builder()) {
        connectTimeout(CONNECTION_TIMEOUT, TimeUnit.SECONDS)
        readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
        pingInterval(pingIntervalSec, TimeUnit.SECONDS)

//        if (BuildConfig.DEBUG)
            addInterceptor(loggingInterceptor)
        authenticator(tokenAuthenticator)
        build()
    }

Authenticator:

class AuthAccessTokenAuthenticator(
    private val oAuthRepo: OAuthRepo
) : Authenticator {

    override fun authenticate(route: Route?, response: Response): Request? {
        return runBlocking {
            val accessTokenData = oAuthRepo.getAuthAccessToken()
                .let {
                    if (it is ResponseState.Success)
                        it.result
                    else
                        null
                }

            if (isRequestWithAccessToken(response)
                    || accessTokenData == null)
                return@runBlocking null

            return@runBlocking newRequestWithAccessToken(
                response.request,
                accessTokenData.accessToken
            )
        }
    }

    private fun isRequestWithAccessToken(response: Response): Boolean {
        val header = response.request.header("Authorization")
        return header != null && header.startsWith("Bearer")
    }

    private fun newRequestWithAccessToken(request: Request, accessToken: String): Request {
        return request.newBuilder()
            .header("Authorization", "Bearer $accessToken")
            .build()
    }

}

You can see that i added authenticator to my OkHttpClient but client.ws doesn't add an authorization token to my requests and I have to manually add it via header (). Why?

client.ws(host = "host",
                path = "path",
                request = {
                    Timber.d("Sockets -> Building socket HttpRequest")
                    url.protocol = URLProtocol.WS
                    url.port = 80
                    Timber.d("Sockets -> Starting socket connection")
                    //workaround for authorization header("Authorization", "token")
                })
``

ResponseException is no longer serializable starting from 1.4.0 (breaking change)

https://youtrack.jetbrains.com/issue/KTOR-1552

Stack trace:

io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
java.io.NotSerializableException: io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)
	at ExceptionsTest.serialize(ExceptionsTest.kt:26)
	at ExceptionsTest.testResponseExceptionSerializable(ExceptionsTest.kt:17)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)

Here is a test to reproduce the issue:

import io.ktor.client.call.*
import io.ktor.client.features.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.util.date.*
import io.ktor.utils.io.*
import java.io.*
import kotlin.coroutines.*
import kotlin.test.*

class ExceptionsTest {

    @Test
    fun testResponseExceptionSerializable() {
        val exception = createResponseException()

        val serialized = serialize(exception)
        val deserialized = deserialize(serialized)

        deserialized as ResponseException
    }

    private fun serialize(obj: Any): ByteArray {
        val b = ByteArrayOutputStream()
        val o = ObjectOutputStream(b)
        o.writeObject(obj)
        return b.toByteArray()
    }

    private fun deserialize(bytes: ByteArray): Any? {
        val b = ByteArrayInputStream(bytes)
        val o = ObjectInputStream(b)
        return o.readObject()
    }
}

private fun createResponseException(): ResponseException = ResponseException(object : HttpResponse() {
    override val call: HttpClientCall
        get() = TODO("Not yet implemented")
    override val status: HttpStatusCode
        get() = TODO("Not yet implemented")
    override val version: HttpProtocolVersion
        get() = TODO("Not yet implemented")
    override val requestTime: GMTDate
        get() = TODO("Not yet implemented")
    override val responseTime: GMTDate
        get() = TODO("Not yet implemented")
    override val content: ByteReadChannel
        get() = TODO("Not yet implemented")
    override val headers: Headers
        get() = TODO("Not yet implemented")
    override val coroutineContext: CoroutineContext
        get() = TODO("Not yet implemented")

    override fun toString(): String = "FakeCall"
}, cachedResponseText = "Fake text")

Ktor 1.4.1 fails to upload file (submit form) when logging level = LogLevel.BODY

Overload resolution ambiguity: HttpClient.ws

Content-type header is missing in Request when using LogLevel.BODY

Jetty client engine with high concurrency randomly freezes, throws NPEs, ISEs or EofExceptions

https://youtrack.jetbrains.com/issue/KTOR-1120

Exception in thread "main" java.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at org.eclipse.jetty.io.SelectorManager.connect(SelectorManager.java:193)
	at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:409)
	at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:382)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt:61)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt)
	at io.ktor.client.engine.jetty.UtilsKt.withPromise(utils.kt:14)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.connect(JettyHttpRequest.kt:59)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.executeRequest(JettyHttpRequest.kt:31)
	at io.ktor.client.engine.jetty.JettyHttp2Engine.execute(JettyHttp2Engine.kt:38)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more

import io.ktor.client.*
import io.ktor.client.engine.jetty.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*

suspend fun main() = withContext(Dispatchers.Default) {
   HttpClient(Jetty) {
      expectSuccess = false

      install(HttpTimeout) {
         connectTimeoutMillis = 2_000
         socketTimeoutMillis = 2_000
      }

      engine {
         threadsCount = 200
      }
   }.use { client ->
      coroutineScope {
         repeat(5) {
            repeat(200) {
               launch {
                  client.get<HttpResponse>("https://esi.evetech.net/latest/markets/10000040/orders/?page=1")
                     .readText()
                     .let { println(it.length) }
               }
            }
         }
      }
   }

   println()
   println("Done")
}

Native: ResponseObserver hangs coroutine indefinitely

CertificatePinner buildErrorMessage outputs CertificatesInfo reference and variable name (HASH_ALGORITHM_SHA_256) instead of value

Client: InvalidCacheStateException when Vary header differs for 200 and 304 responses

Ktor Client HTTP Cache for invalid or blank Expires header

https://youtrack.jetbrains.com/issue/KTOR-364

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1892

Ktor Version and Engine Used (client or server and name)
1.3.2, client, io.ktor.client.features.cache.HttpCache

Describe the bug
If a server returns a blank or an invalid Expires HTTP header, the ktor client crashes if the feature HttpCache is enabled install(HttpCache)

To Reproduce
Steps to reproduce the behavior:

Write the following:

val articleUrl = Url("https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503")
val newsClient = HttpClient(Apache) {
    install(HttpCache)

    install(UserAgent) {
        agent = "Mozilla/5.0"
    }
}
try {
    val call: HttpStatement = newsClient.get(articleSimilarityRequest.articleUrl)
    val response: HttpResponse = call.receive()
    val requestUrl = response.request.url.toString()
    val rawHtml = response.readText()
    println(rawHtml)
} catch (cause: Throwable) {
    println("Unable to download article")
} finally {
    newsClient.close()
}

If you open the URL https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503 in the browser, you can see that the Expires response header is blank
An exception is thrown if you run the code:

2020-05-23 14:41:24.124 [DefaultDispatcher-worker-2 @request#48] ERROR a.s.api.util.service.DownloadService - Unable to download article
java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	(Coroutine boundary)
	at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:123)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:116)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:89)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
	at app.shwoo.api.util.service.DownloadService.downloadArticle$suspendImpl(DownloadService.kt:101)
	at app.shwoo.api.server.web.SimilarityResourceKt$similarity$1$4.invokeSuspend(SimilarityResource.kt:38)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
	at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:292)
	at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
	at io.ktor.server.testing.TestApplicationEngine$handleRequest$pipelineJob$1.invokeSuspend(TestApplicationEngine.kt:290)
Caused by: java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior
It seems that the problem occurs in https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/features/cache/HttpCacheEntry.kt#L84. If the Expires header is blank an exception is thrown. The expected behavior is that the application handles the cache as expired if an invalid date is provided in the HTTP Expires header. Like described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires

NoTransformationFoundException when using JsonFeature and connecting through a proxy server

https://youtrack.jetbrains.com/issue/KTOR-955

Steps of reproduction:

Add windows or macOS targets to the Gradle build file if required.
Run ./gradlew runDebugExecutableLinuxX64 (replace with your target)

As a result, the following exception is thrown:

> Task :runDebugExecutableLinuxX64 FAILED
Uncaught Kotlin exception: io.ktor.client.call.NoTransformationFoundException: No transformation found: class io.ktor.utils.io.ByteChannelNative -> class Mine
with response from https://httpbin.org/get:
status: 200 OK
response headers: 

    at kfun:kotlin.Throwable#<init>(){} (0x306ebe)
    at kfun:kotlin.Exception#<init>(){} (0x3007b7)
    at kfun:kotlin.RuntimeException#<init>(){} (0x300377)
    at kfun:kotlin.UnsupportedOperationException#<init>(){} (0x300bf7)
    at kfun:io.ktor.client.call.NoTransformationFoundException#<init>(io.ktor.client.statement.HttpResponse;kotlin.reflect.KClass<*>;kotlin.reflect.KClass<*>){} (0x579a53)
    at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$12#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? (0x578b21)
    at kfun:io.ktor.client.call.HttpClientCall#receive(io.ktor.client.call.TypeInfo){}kotlin.Any (0x578fc6)
    at kfun:$main$lambda-2COROUTINE$0.invokeSuspend#internal (0x640c1f)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x322d4f)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
Unfinished workers detected, 1 workers leaked!
Use `Platform.isMemoryLeakCheckerActive = false` to avoid this check.
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
    at kfun:kotlinx.coroutines.DispatchedTask#run(){} (0x4114a6)
    at kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long (0x3e9ace)
    at kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} (0x423639)
    at kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal (0x422de0)
    at kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 (0x42223b)
    at kfun:kotlinx.coroutines#runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 (0x422741)
    at kfun:#main(){} (0x63e8cc)
    at Konan_start (0x641839)
    at Init_and_run_start (0x66cdcb)
    at __libc_start_main (0x7f0318cb20b3)
    at  (0x2b8029)
    at  ((nil))

Here is the related Github issue.

CIO may not work correctly with proxy for https requests

https://youtrack.jetbrains.com/issue/KTOR-1306

Steps:

privoxy setup:

$ cat  ./privoxyconfig 
confdir /etc/privoxy
temporary-directory /tmp
logdir /var/log/privoxy
actionsfile match-all.action # Actions that are applied to all sites and maybe overruled later on.
actionsfile default.action   # Main actions file
actionsfile user.action      # User customizations
filterfile default.filter
filterfile user.filter      # User customizations
logfile privoxy.log
debug     1 # Log the destination for each request Privoxy let through. See also debug 1024.
debug  1024 # Log the destination for requests Privoxy didn't let through, and the reason why.
debug  4096 # Startup banner and warnings
debug  8192 # Non-fatal errors
listen-address  0.0.0.0:8118
toggle  1
enable-remote-toggle  0
enable-remote-http-toggle  0
enable-edit-actions 0
enforce-blocks 1
buffer-limit 4096
enable-proxy-authentication-forwarding 0
trusted-cgi-referer http://www.example.org/
forwarded-connect-retries  0
accept-intercepted-requests 0
allow-cgi-request-crunching 0
split-large-forms 0
tolerate-pipelining 1
socket-timeout 300

$ cat ./Dockerfile 
FROM alpine
RUN apk add --no-cache privoxy=3.0.28-r0
COPY privoxyconfig /etc/privoxy/config
EXPOSE 8118
CMD privoxy --no-daemon /etc/privoxy/config

$ docker build . -t privoxy
$ docker run --rm -p 127.0.0.1:8118:8118 privoxy

set proxy as described in https://ktor.io/docs/proxy.html with

fun main(args: Array<String>) {
    val client = HttpClient(CIO) {
        install(UserAgent) {
            agent = "Space Unfurl Bot"
        }

        install(ContentEncoding) {
            gzip()
        }

        engine {
            proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("127.0.0.1", 8118))
        }
    }
    val r = runBlocking(CoroutineName("Get Metadata")) {
        client.get<HttpStatement>("https://example.com").execute { call ->
            "response code: ${call.status.description}"

        }
    }
    println(r)
}

Request:
[X] https://example.com CIO hangs for a long time and returns java.lang.IllegalArgumentException: Invalid TLS record type code: 72
[X] http://ya.ru same result (ya.ru redirects to https)
[V] http://example.com returns page content
[V] Replace CIO with Apache - any requests work perfectly fine.

KTOR Default url (hostname)

Apache HttpClient engine: connection exception does not propagate, receiving JobCancellationException

IncorrectDereferenceException thrown when ktor request call from other thread

Core

java.lang.NoSuchMethodError: java.nio.ByteBuffer.limit(I)Ljava/nio/ByteBuffer when Ktor is built with JDK 9+

https://youtrack.jetbrains.com/issue/KTOR-1398

To reproduce:

Run ./gradlew publishToMavenLocal in the root directory of the Ktor framework repository on the latest master. Use JDK 9+ for a build.
Use the 1.4.3-SNAPSHOT (at the time of writing) version of Ktor in a project with Maven local repository.
Build and run the following code with JDK 8:

suspend fun main(args: Array<String>) {
    val client = HttpClient(CIO)
    client.get<String>("http://example.com")
}

As the result unexpected exception will be thrown:

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.limit(I)Ljava/nio/ByteBuffer;
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt:106)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt)
	at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
	at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2606)
	at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannel.setupStateForWrite$ktor_io(ByteBufferChannel.kt:211)
	at io.ktor.utils.io.ByteBufferChannel.copyDirect$ktor_io(ByteBufferChannel.kt:2758)
	at io.ktor.utils.io.ByteReadChannelJVMKt.copyTo(ByteReadChannelJVM.kt:276)
	at io.ktor.client.engine.cio.UtilsKt$withoutClosePropagation$2.invokeSuspend(utils.kt:173)
	at io.ktor.client.engine.cio.UtilsKt$withoutClosePropagation$2.invoke(utils.kt)
	at io.ktor.utils.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:129)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.limit(I)Ljava/nio/ByteBuffer;
	at io.ktor.utils.io.ByteBufferChannel.prepareBuffer(ByteBufferChannel.kt:193)
	at io.ktor.utils.io.ByteBufferChannel.setupStateForWrite$ktor_io(ByteBufferChannel.kt:242)
	at io.ktor.utils.io.ByteBufferChannel.copyDirect$ktor_io(ByteBufferChannel.kt:2758)
	at io.ktor.utils.io.ByteReadChannelJVMKt.copyTo(ByteReadChannelJVM.kt:276)
	at io.ktor.utils.io.ByteReadChannelKt.copyAndClose(ByteReadChannel.kt:267)
	at io.ktor.utils.io.ByteReadChannelKt.copyAndClose$default(ByteReadChannel.kt:266)
	at io.ktor.network.sockets.TimeoutExceptionsCommonKt$mapEngineExceptions$2.invokeSuspend(TimeoutExceptionsCommon.kt:61)
	at io.ktor.network.sockets.TimeoutExceptionsCommonKt$mapEngineExceptions$2.invoke(TimeoutExceptionsCommon.kt)
	... 7 more

You can find a similar issue here.

Significant performance penalty with socket write() and file read() on JVM

URLBuilder: Move Default Values to build() function

ktor-io: JVM shared function decrease performance starting from 1.4.0

Dependency for ktor-network 1.4.1 and 1.4.0 doesn't work on Kotlin/Multiplatform

https://youtrack.jetbrains.com/issue/KTOR-1270

I tried to apply the ktor-network dependency to my commonMain source set dependencies, but I received an error saying that the dependency was not found. I also tried to apply it to the dependencies of the sourceSet commonMain and it seems to work there.

Repositories applied: jcenter, mavenCentral
build.gradle.kts:

plugins {
    kotlin("multiplatform") version "1.4.10"
    kotlin("plugin.serialization") version "1.4.10"
}

val ktorVersion = "1.4.1"
val coroutinesVersion = "1.4.1"
val serializationVersion = "1.0.1"

allprojects {

    plugins.apply("org.jetbrains.kotlin.multiplatform")
    plugins.apply("org.jetbrains.kotlin.plugin.serialization")

    repositories {
        mavenCentral()
        jcenter()
    }

    kotlin {

        jvm {
            compilations.all {
                kotlinOptions.jvmTarget = "1.8"
            }
            testRuns["test"].executionTask.configure {
                useJUnitPlatform()
            }
        }
        val hostOs = System.getProperty("os.name")
        val isMingwX64 = hostOs.startsWith("Windows")

        val nativeTarget = when {
            hostOs == "Mac OS X" -> macosX64("native")
            hostOs == "Linux" -> linuxX64("native")
            isMingwX64 -> mingwX64("native")
            else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
        }

        sourceSets {
            val commonMain by getting {
                dependencies {
                    implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:$serializationVersion")
                    implementation("io.ktor:ktor-network:$ktorVersion")
                    implementation("io.ktor:ktor-io:$ktorVersion")
                    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion")
                }
            }
            val commonTest by getting {
                dependencies {
                    implementation(kotlin("test-common"))
                    implementation(kotlin("test-annotations-common"))
                }
            }
            val jvmMain by getting
            val jvmTest by getting {
                dependencies {
                    implementation(kotlin("test-junit5"))
                    implementation("org.junit.jupiter:junit-jupiter-api:5.6.0")
                    runtimeOnly("org.junit.jupiter:junit-jupiter-engine:5.6.0")
                }
            }
            val nativeMain by getting
            val nativeTest by getting
        }
    }

}

error screenshot:

CIO client engine randomly runs into infinite loop under high load

https://youtrack.jetbrains.com/issue/KTOR-1126

Stuck in while (!closed) in io.ktor.network.selector.ActorSelectorManager.process().
Probably related to KTOR-717.
Likely a duplicate of KTOR-531.

I also randomly get this:

Exception in thread "main" java.net.BindException: Can't assign requested address
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Net.java:503)
	at java.base/sun.nio.ch.Net.connect(Net.java:492)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:751)
	at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:31)
	at io.ktor.network.sockets.ConnectUtilsJvmKt.connect(ConnectUtilsJvm.kt:19)
	at io.ktor.network.sockets.TcpSocketBuilder.connect(TcpSocketBuilder.kt:38)
	at io.ktor.client.engine.cio.ConnectionFactory.connect(ConnectionFactory.kt:24)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invokeSuspend(Endpoint.kt:159)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invoke(Endpoint.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:120)
	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:83)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:167)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:95)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more

Repro:

import io.ktor.application.*
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import kotlin.random.*
import kotlin.time.*
import kotlinx.coroutines.*

@OptIn(ExperimentalTime::class)
suspend fun main() = withContext(Dispatchers.Default) {
    startServer()
    delay(1_000)

    val batchSize = 100
    val threadCount = 1

    HttpClient(CIO) {
        engine {
            threadsCount = threadCount
        }
    }.use { client ->
        repeat(Int.MAX_VALUE) { i ->
            val time = measureTime {
                coroutineScope {
                    repeat(batchSize) {
                        launch {
                            client.get<HttpResponse>("http://localhost:8080").readText()
                        }
                    }
                }
            }

            println("${i + 1}: $time")
        }
    }
}

private fun startServer() {
    embeddedServer(Netty, port = 8080) {
        val random = Random(0)
        val data = buildString(100_000) { repeat(100_000) { append(random.nextInt(0, 10)) } }

        routing {
            get("/") {
                call.respondText(data, ContentType.Text.Plain)
            }
        }
    }.start()
}

Docs

The kotlinx.serialization feature documentation is using deprecated API

[docs] Wrong shadow plugin version in Fat JAR docs

Generator

"Using userNameRealmPasswordDigestProvider is an error" compilation error of a generated project

Generator frontend: "TypeError: (0 , _components.default) is not a function" when non supported language specified in markdown

Fix height of the right panel on the new generator website.

Ktor generator produces erroneous build.gradle.kts if KTS is chosen

Infrastructure

Fix project configuration in IDEA on mingw host

ktor-server-test-host indirectly depends on logback-classic

Add artifact dependency to release configuration

CI does not trigger build on PR to samples repo

Invalid publication 'kotlinMultiplatform' for task :publishKotlinMultiplatformPublicationToMavenLocal on master

IntelliJ IDEA Plugin

Plugin: "Could not find io.ktor:ktor-client-http-timeout" when building project created with http-timeout feature

Plugin: the Ktor project type tooltip in the New Project wizard contains text used for Java projects

Test compilation errors when project created via plugin with "Mock HttpClient Engine"

No changelog for 1.4.0 and 1.4.1 on the plugin page in IDE Marketplace

Samples

Samples: native-client compilation fails on compileKotlinMetadata

Server

HTTP/2 push fails with Netty engine

https://youtrack.jetbrains.com/issue/KTOR-800

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/350

The problem reproduces with http2-push from https://github.com/ktorio/ktor-samples

Run ./gradlew :http2-push:run
Navigate to https://localhost:8443/

Resulting exception:

2018-03-07 14:47:07.089 [nettyWorkerPool-3-2] WARN  i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
2018-03-07 14:47:07.093 [nettyWorkerPool-3-2] WARN  i.n.util.concurrent.DefaultPromise - An exception was thrown by io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete()
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
        at io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete(NettyHttp2Handler.kt:109)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
        at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163)
        at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:108)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        ... 7 common frames omitted

Cannot receive content via jackson negotiator since 1.4.2

https://youtrack.jetbrains.com/issue/KTOR-1286

It seems like this commit broke the Jackson content negotiator, now when calling call.receive() the following exception occurs:

java.lang.IllegalStateException: Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
	at io.ktor.utils.io.jvm.javaio.BlockingKt.ensureParkingAllowed(Blocking.kt:302)
	at io.ktor.utils.io.jvm.javaio.BlockingKt.access$ensureParkingAllowed(Blocking.kt:1)
	at io.ktor.utils.io.jvm.javaio.InputAdapter.<init>(Blocking.kt:30)
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream(Blocking.kt:20)
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream$default(Blocking.kt:20)
	at io.ktor.jackson.JacksonConverter.convertForReceive(JacksonConverter.kt:55)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:175)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:110)
	at net.octyl.ourtwobe.api.ApiServerKt$module$7$2$1$1.invokeSuspend(ApiServer.kt:232)
	at net.octyl.ourtwobe.api.ApiServerKt$module$7$2$1$1.invoke(ApiServer.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.routing.Routing.executeResult(Routing.kt:148)
	at io.ktor.routing.Routing.interceptor(Routing.kt:35)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:100)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:101)
	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:194)
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:100)
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:140)
	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:55)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:38)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:832)

Unexpected exception when using Session feature: "Using blocking primitives on this dispatcher is not allowed"

https://youtrack.jetbrains.com/issue/KTOR-1452

kotlin: 1.4.20
kotlinx.coroutines: 1.4.2

Sessions feature is configured in a rather standard manner:

        install(Sessions) {
            cookie<UserIdPrincipal>(
                Cookies.SESSION_ID,
                storage = directorySessionStorage(sessionStorageRootDir)
            ) {
                cookie.path = "/"
                cookie.httpOnly = true
                cookie.extensions["SameSite"] = "Lax"
            }
        }

When it is called during a pipeline execution, an unexpected exception is thrown.

java.lang.IllegalStateException: Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
	at io.ktor.utils.io.jvm.javaio.BlockingKt.ensureParkingAllowed(Blocking.kt:302) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.access$ensureParkingAllowed(Blocking.kt:1) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.InputAdapter.<init>(Blocking.kt:30) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream$default(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.DirectoryStorage.read(DirectoryStorage.kt:46) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invokeSuspend(Cache.kt:82) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invoke(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2$1.invokeSuspend(Cache.kt:38) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	(Coroutine boundary) ~[?:?]
	at kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:101) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.sessions.ReferenceCache.getOrCompute$suspendImpl(Cache.kt:86) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage.read(CacheStorage.kt:20) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionTrackerById.load(SessionTrackerById.kt:34) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:215) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:259) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:110) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:102) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:101) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:142) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118) ~[ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:118) ~[ktor-server-netty-jvm-1.4.3.jar:1.4.3]
Caused by: java.lang.IllegalStateException: Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
	at io.ktor.utils.io.jvm.javaio.BlockingKt.ensureParkingAllowed(Blocking.kt:302) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.access$ensureParkingAllowed(Blocking.kt:1) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.InputAdapter.<init>(Blocking.kt:30) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.utils.io.jvm.javaio.BlockingKt.toInputStream$default(Blocking.kt:20) ~[ktor-io-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.DirectoryStorage.read(DirectoryStorage.kt:46) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invokeSuspend(CacheStorage.kt:15) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage$referenceCache$1.invoke(CacheStorage.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invokeSuspend(Cache.kt:82) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache$container$1.invoke(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2$1.invokeSuspend(Cache.kt:38) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.4.20.jar:1.4.20-release-308 (1.4.20)]
	at kotlinx.coroutines.internal.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:342) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:30) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable$default(Cancellable.kt:27) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt:91) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.async(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async$default(Builders.common.kt:84) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.async$default(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.sessions.BaseCache$getOrCompute$2.apply(Cache.kt:37) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseCache$getOrCompute$2.apply(Cache.kt:31) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_251]
	at io.ktor.sessions.BaseCache.getOrCompute(Cache.kt:36) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache.getOrCompute$suspendImpl(Cache.kt:86) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.ReferenceCache.getOrCompute(Cache.kt) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.BaseTimeoutCache.getOrCompute(Cache.kt:163) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.CacheStorage.read(CacheStorage.kt:20) ~[ktor-server-sessions-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionTrackerById.load(SessionTrackerById.kt:34) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:215) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:63) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt) ~[ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:110) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:102) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:194) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:101) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:142) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt) [ktor-server-core-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118) [ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt) [ktor-server-host-common-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) [ktor-utils-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:118) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source) [kotlinx-coroutines-core-jvm-1.4.2.jar:?]
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:43) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:34) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [netty-transport-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:216) [ktor-server-netty-jvm-1.4.3.jar:1.4.3]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.54.Final.jar:4.1.54.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251]

Jackson: Crash when sending large responses in 1.4.2

https://youtrack.jetbrains.com/issue/KTOR-1369

Hi there,

The error logged is:

Exception in thread "eventLoopGroupProxy-3-1" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@2b5e16e4, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@114e04cb]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
        at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2258)
        at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:929)
        at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1953)
        at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:305)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
        at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
        at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)

→ http localhost:8080/small
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
transfer-encoding: chunked

{
    "string": "small string"
}

→ http localhost:8080/large
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
transfer-encoding: chunked


http: error: ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

Please let me know if you need anything else.

Thanks,
Skye

Micrometer metrics: scan of random URLs of a server exhausts available values per tag in influx

"Wrong HEX escape": gracefully handle invalid URLs

https://youtrack.jetbrains.com/issue/KTOR-1308

We recently got some alerts in our webserver because ktor threw an uncaught exception. Ideally these are handled and a 400 is sent back.

Stacktrace:

at io.ktor.http.CodecsKt.decodeImpl (Codecs.kt:197)
at io.ktor.http.CodecsKt.decodeScan (Codecs.kt:146)
at io.ktor.http.CodecsKt.decodeURLPart (Codecs.kt:140)
at io.ktor.http.CodecsKt.decodeURLPart$default (Codecs.kt:139)
at io.ktor.routing.RoutingResolveContext.parse (RoutingResolve.kt:70)
at io.ktor.routing.RoutingResolveContext.<init> (RoutingResolve.kt:49)
at io.ktor.routing.Routing.interceptor (Routing.kt:31)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend (Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke (Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend (ContentNegotiation.kt:107)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke (ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend (DefaultEnginePipeline.kt:120)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke (DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend (NettyApplicationCallHandler.kt:40)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke (NettyApplicationCallHandler.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched (Undispatched.kt:55)
at kotlinx.coroutines.CoroutineStart.invoke (CoroutineStart.kt:111)
at kotlinx.coroutines.AbstractCoroutine.start (AbstractCoroutine.kt:158)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch (Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch (Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest (NettyApplicationCallHandler.kt:30)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead (NettyApplicationCallHandler.kt:24)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600 (AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run (AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)

Netty HTTP/2 HEAD response hangs

Sessions + SSL (Netty)

https://youtrack.jetbrains.com/issue/KTOR-1292

edit: https://github.com/dm6801/ktor_sessions_ssl

when trying to use Session feature with cookies

    install(Sessions) {
        cookie<MySession>("MY_SESSION")
    }

accessing https:/localhost:443/ throws:

java.lang.UnsupportedOperationException: null
	(Coroutine boundary)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:55)
	(Coroutine creation stacktrace)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:188)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:38)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:821)
Caused by: java.lang.UnsupportedOperationException: null
	at io.ktor.server.netty.http2.NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64)
	at io.ktor.sessions.SessionTransportCookie.receive(SessionTransportCookie.kt:28)
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:204)
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:59)
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)

relevant line NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64):

override val cookies: RequestCookies
    get() = throw UnsupportedOperationException()

http://localhost:8080/ - works

Sensible defaults for compression feature

Setting multiple cache control directives is impossible with current API

Netty: NumberFormatException when make a request via HTTP/2 without explicit port

https://youtrack.jetbrains.com/issue/KTOR-1091

Hello.

There is a bug with Http2LocalConnectionPoint. If there is no port specified in authority then it tries to parse domain as port.
I've created a project to reproduce the bug: https://github.com/Vlad8161/netty-bug-sample

To reproduce it run the server on 443 port and access it without explicit port specifying.

Stack Trace:

java.lang.NumberFormatException: For input string: "localhost"
     	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 	at java.lang.Integer.parseInt(Integer.java:580)
 	at java.lang.Integer.parseInt(Integer.java:615)
 	at io.ktor.server.netty.http2.Http2LocalConnectionPoint.getPort(Http2LocalConnectionPoint.kt:31)
 	at io.ktor.features.HSTS.intercept(HSTS.kt:87)
 	at io.ktor.features.HSTS$Feature$install$1.invokeSuspend(HSTS.kt:101)
 	at io.ktor.features.HSTS$Feature$install$1.invoke(HSTS.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:101)
 	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
 	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:189)
 	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:100)
 	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:140)
 	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
 	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
 	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
 	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:121)
 	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
 	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
 	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:54)
 	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
 	at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:182)
 	at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source)
 	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:145)
 	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
 	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
 	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37)
 	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
 	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
 	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 	at java.lang.Thread.run(Thread.java:748)

The problem is at Http2LocalConnectionPoint:31

I think is would be better to do something like this:

nettyHeaders.authority()?.toString()?.substringAfter(":", if (scheme == "https") "443" else "80")?.toInt()

because often local port information is not enough

Thanks in advance :)

Class cast exception on web-socket cancelation

https://youtrack.jetbrains.com/issue/KTOR-349

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1929

Ktor Version and Engine Used (client or server and name)
ktor-cio server 1.3.2

Describe the bug
I am getting an internal error when force-closing server with opened web-socket connection
Here is the stack trace:

13:32:20.839 [DefaultDispatcher-worker-4] ERROR io.ktor.server.cio.HttpServer - Unhandled exception caught for CoroutineName(http-pipeline-writer)
kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[ExperimentalCoroutineDispatcher@45e6d6a4[scheduler = DefaultDispatcher@6341301e[Pool Size {core = 10, max = 1024}, Worker States {CPU = 6, blocking = 0, parked = 8, dormant = 0, terminated = 0}, running workers queues = [0c, 0c, 0c, 0c, 0c, 0c], global CPU queue size = 5, global blocking queue size = 0, Control State {created workers= 14, blocking tasks = 0, CPUs acquired = 6}]], Continuation at io.ktor.server.cio.backend.ServerPipelineKt$startServerConnectionPipeline$1$outputsActor$1.invokeSuspend(ServerPipeline.kt:49)@2482cf33]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	... 4 common frames omitted

CORS - request with non-simple content type not failing

Scheme instead of version in OriginConnectionPoint

Weird Ktor internal exception

https://youtrack.jetbrains.com/issue/KTOR-2358

Hello!

Exception in thread "eventLoopGroupProxy-3-2" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@1398676, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@2a2565a2]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
        at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
        at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
        at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2258)
        at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:929)
        at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1953)
        at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:305)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException

This exception regularly appears in the output of my Ktor Webserver. Most of the time its harmless, but every 24 hours or so, it seems to be turning my webserver unresponsive (The requests arrive, but nothing gets sent back apparently), which forces me to restart the program. I am using version 8.0.282 of Azul Zulu OpenJDK, version 1.4.10 of Kotlin and version 1.4.2 of Ktor on Ubuntu 18.04. Help would be very much appreciated.

Best,
Darki

Netty: Thymeleaf java.lang.ClassCastException when template file is more than 4088 bytes in size

https://youtrack.jetbrains.com/issue/KTOR-1373

Created a project via start.ktor.io
replaced the content of the index.html file ( project to reproduce )
./gradlew run and open browser http://0.0.0.0:8080/html-thymeleaf
got:

Exception in thread "eventLoopGroupProxy-3-1 @response-pipeline#4" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@4d744708, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@600c2262]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
	at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2258)
	at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:929)
	at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1953)
	at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:305)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')

	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	... 22 more

Fatal exception in ktor

https://youtrack.jetbrains.com/issue/KTOR-1342

I'm not sure why or where this happens, but we attempted to upgrade ktor to 1.4.2 and we got this error, I've downgraded to 1.3.2 which seems to have fixed it. I have two different stacktraces.

kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@6ed39d43, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@dc068af]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core (DispatchedTask.kt:144)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:115)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch (Blocking.kt:296)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith (DispatchedContinuation.kt:184)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith (Blocking.kt:163)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:106)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch (Blocking.kt:296)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith (DispatchedContinuation.kt:184)
at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith (CancellableReusableContinuation.kt:93)
at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp (ByteBufferChannel.kt:2258)
at io.ktor.utils.io.ByteBufferChannel.bytesRead (ByteBufferChannel.kt:929)
at io.ktor.utils.io.ByteBufferChannel.consumed (ByteBufferChannel.kt:1953)
at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend (NettyResponsePipeline.kt:305)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run (NettyApplicationEngine.kt:215)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)
at java.lang.Thread.run (Thread.java:834)

otlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@6ed39d43, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@2f25fce0]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core (DispatchedTask.kt:144)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:115)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch (Blocking.kt:296)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith (DispatchedContinuation.kt:184)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith (Blocking.kt:163)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:106)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch (Blocking.kt:296)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith (DispatchedContinuation.kt:184)
at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith (CancellableReusableContinuation.kt:93)
at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp (ByteBufferChannel.kt:2258)
at io.ktor.utils.io.ByteBufferChannel.bytesRead (ByteBufferChannel.kt:929)
at io.ktor.utils.io.ByteBufferChannel.consumed (ByteBufferChannel.kt:1953)
at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend (NettyResponsePipeline.kt:305)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run (NettyApplicationEngine.kt:215)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)
at java.lang.Thread.run (Thread.java:834)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.internal.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.internal.DispatchedContinuation are in unnamed module of loader 'app')
at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation (CoroutineDispatcher.kt:104)
at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted (ContinuationImpl.kt:118)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:39)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:104)

NoSuchMethodError: No virtual method toPath() in android API < 26

Fatal exception returning payloads via ContentNegotiation feature in JVM/Netty

https://youtrack.jetbrains.com/issue/KTOR-1322

Cannot recreate in 1.4.1 or earlier versions
Running on JVM (AdoptOpenJDK 11.0.7+10)
Using Netty-based application engine
Does not appear to be specifically related to JacksonConverter or ContentNegotiation code, but in how that work is handed off to threads

When using the JacksonConverter with the ContentNegotiation feature, payloads 4kb or larger throw an exception and ultimately return incomplete data (manifesting as Failed to load resource: net::ERR_INCOMPLETE_CHUNKED_ENCODING in Firefox - different messages, same symptom in other clients).

After some debugging, it appears all Jackson bytestream work eventually goes through BlockingBridge.withBlocking() and always ends up with safeToRunInPlace evaluating to true. In Blocking.rendezvous(), smaller than 4kb payloads are on threads named DefaultDispatcher-worker-* which can successfully park(), where larger payloads are on threads named eventLoopGroupProxy-*-* which have a Parking impl defined as ProhibitParking. This is new behavior introduced in 1.4.2 as part of this commit: https://github.com/ktorio/ktor/commit/0dfada8cd8e438d724ce77e9cd0d8b09a6b8c6dc#diff-9004ca70aa325dbcfa7d3ce09d193306f9f33bfd9c7c7732875f008e294feb6d and throws the following:

java.lang.UnsupportedOperationException: Parking is prohibited on this thread. Most likely you are using blocking operation on the wrong thread/dispatcher that doesn't allow blocking. Consider wrapping you blocking code withContext(Dispatchers.IO) {...}.

This exception seems to lead to the coroutine ending up as a CompletedContinuation, which is then cast to DispatchedContinuation in CoroutineDispatcher.releaseInterceptedContinuation(). This is what leads to the final representation of the stack trace that I see in logs:

Exception in thread "eventLoopGroupProxy-5-1" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[UnsafeBlockingTrampoline@5f68303f, Continuation at io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1.loop(Blocking.kt:311)@444cf2bb]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:144)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:115)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter$end$1.resumeWith(Blocking.kt:163)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:296)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:184)
	at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
	at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2258)
	at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:929)
	at io.ktor.utils.io.ByteBufferChannel.consumed(ByteBufferChannel.kt:1953)
	at io.ktor.server.netty.cio.NettyResponsePipeline$processBodyFlusher$2.invokeSuspend(NettyResponsePipeline.kt:305)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.internal.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.internal.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:104)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	... 23 more

A rather simple reproduction is available at https://github.com/rhurkes/ktor-playground/commit/989b512bd7209735fd4d223b839d20d307c533c3

NOTE: I did notice that stepping through code and flipping back and forth between payloads on either side of the 4kb line I could occasionally fail to reproduce this issue. There's definitely some magic in the dispatching that isn't 100% fully deterministic.

Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.

Make Jackson serialization stream response

CORS denies simple request where the content type includes a character encoding

Autoreload embeddedServer configuration fails, if a module has default parameters

https://youtrack.jetbrains.com/issue/KTOR-1334

Ktor Version

1.4.2

Ktor Engine

Netty

JVM Version, Operating System and Relevant Context

JDK 11, Manjaro

Problem

Project is generated by Ktor IDEA plugin, which if test modules are included, generates a module with default parameters.
It fails with Module function provided as lambda cannot be unlinked for reload, if module has default parameters.
Using method reference as in the example, does not make it work.

Example:

fun main(args: Array<String>): Unit {
    embeddedServer(
            Netty,
            port = 8080,
            host = "0.0.0.0",
            watchPaths = listOf("some"),
            module = Application::module,
    ).start(true)
}

fun Application.module(testing: Boolean = false): Unit {
    routing {
        get("/") {
            call.respondText { "Hello" }
        }
    }
}

Running Result

Exception in thread "main" java.lang.RuntimeException: Module function provided as lambda cannot be unlinked for reload
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.<init>(ApplicationEngineEnvironmentReloading.kt:58)
	at io.ktor.server.engine.ApplicationEngineEnvironmentBuilder.build(ApplicationEngineEnvironment.kt:105)
	at io.ktor.server.engine.ApplicationEngineEnvironmentKt.applicationEngineEnvironment(ApplicationEngineEnvironment.kt:44)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:55)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:36)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:34)
	at some.ReloadKt.main(reload.kt:10)

It fails here, basically can't cast the default parameter method as KFunction<*>

public class ApplicationEngineEnvironmentReloading(
//...
) : ApplicationEngineEnvironment {
//...

    private val moduleFunctionNames: List<String>? = run {
        val configModules = config.propertyOrNull("ktor.application.modules")?.getList()
        if (watchPatterns.isEmpty()) configModules
        else {
            val unlinkedModules = modules.map {
         // Fails here
                val fn = (it as? KFunction<*>)?.javaMethod
                    ?: throw RuntimeException("Module function provided as lambda cannot be unlinked for reload")
                val clazz = fn.declaringClass
                val name = fn.name
                "${clazz.name}.$name"
            }
            if (configModules == null)
                unlinkedModules
            else
                configModules + unlinkedModules
        }
    //...
}

Workaround

fun Application.module() = module(false)

fun Application.module(testing: Boolean = false): Unit {...}

Test Infrastructure

Replace JUnit with kotlin.test

Other

Create sample with native CIO client and `native-mt` coroutines

Remove copyTo usage from ServerPipeline

Handle failure in reading request body

Fix HTTP2 tests with new versions of java8

Blocking calls (ReentrantLock.lock) on Apache engine dispatcher threads

https://youtrack.jetbrains.com/issue/KTOR-1140

I'm using BlockHound to check if there are any blocking calls within my coroutines that aren't using IO dispatcher.
BlockHound reports the following occasion for Ktor client with Apache engine on thread ktor-apache-dispatcher-worker-4.
Maybe it's a bug, maybe it's a misconfiguration of the dispatcher threads as CPU-bound.

reactor.blockhound.BlockingOperationError: Blocking call! jdk.internal.misc.Unsafe#park
	at java.base/jdk.internal.misc.Unsafe.park(Unsafe.java)
	at java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917)
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240)
	at java.base/java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:267)
	at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:278)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:295)
	at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:377)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129)
	at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:68)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt.sendRequest(ApacheHttpRequest.kt:33)
	at io.ktor.client.engine.apache.ApacheEngine.execute(ApacheEngine.kt:37)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Url(…) doesn't parse non-HTTP-like URL strings correctly

"response.readText()" never returns (probably infinite loop)

Ktor CIO client, connect timeout handling

https://youtrack.jetbrains.com/issue/KTOR-446

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1720

Using the following code (full example at https://github.com/rrva/ktor-client-connection-leak)
Run the main method repeatedly. Sometimes, all futures created will not complete . Why? Expected behavior is that all created futures will complete eventually, so Done with all will be printed.

 class Client(val fooServiceUrl: String) {

    val client = HttpClient(CIO) {
        engine {
            requestTimeout = 500
            endpoint.connectTimeout = 100
            endpoint.connectRetryAttempts = 2
        }
        install(JsonFeature) {
            serializer = KotlinxSerializer(
                Json(
                    JsonConfiguration(
                        isLenient = true,
                        ignoreUnknownKeys = true,
                        serializeSpecialFloatingPointValues = true,
                        useArrayPolymorphism = true
                    )
                )
            )
        }
    }

    suspend fun fetchFoo(ids: List<String>): List<Foo> {
            return client.request {
                header("User-Agent", "Foo")
                parameter("input", ids.joinToString(","))
                url("$fooServiceUrl/foo")
            }
    }

}

@Serializable
data class Foo(val id: String)
val client = Client("http://localhost:9090")

fun fetchFooAsync(ids:List<String>): CompletableFuture<List<Foo>> {
    return CoroutineScope(Job() + Dispatchers.IO + MDCContext()).async {
        try {
            client.fetchFoo(ids)
        } catch (e: Throwable) {
            println(e.message)
            listOf<Foo>(Foo("timeout"))
        }
    }.asCompletableFuture()
}


fun main() {
    HttpServer.create(InetSocketAddress(9090), 1000).apply {

        createContext("/foo") { http ->
            http.responseHeaders.add("Content-type", "application/json")
            http.sendResponseHeaders(200, 0)
            PrintWriter(http.responseBody).use { out ->
                out.println("""[{"id":"1"}]""")
            }
        }

        start()
    }
    println("Start")

    val requests = (1..100).map {
        fetchFooAsync(listOf(it.toString()))
    }
    for (request in requests) {
        println("Waiting for coroutine...")
        println(request.get())
        println("Done waiting")
    }

    println("Done with all")

}

Reproduced on 1.3.1

Extract flaky tests from `master` to the `flaky-tests` branch

Split ktor-server-test-host module

Ktor Client Null Pointer Exception (HTTPS)

TestHttpClientEngine KNPE

Fix configuration if project without VPN and cache

Update kotlin to 1.4.20

NetworkOnMainThreadException when using a ktor mock server

Error: Could not find or load main class {MainClass}

embeddedServer Jetty: java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletResponse

Not able to build iOS framework with Xcode 11.3.1 and ktor version 1.4.2 onwards. However, it works with ktor version 1.4.1

1.4.1

released 24th September 2020

Client

[iOS] InvalidMutabilityException: mutation attempt of frozen

https://youtrack.jetbrains.com/issue/KTOR-1223

ONLY ON iOS, on android work correctly.

I've this class

open class BaseClient(val baseUrl: String) {

    open val serializer = KotlinxSerializer(Json {
        isLenient = true
        ignoreUnknownKeys = true
    })
    open val logLevel = LogLevel.ALL
    open val header = mapOf<String, String>()

    open val client = HttpClient() {
        install(JsonFeature) {
            serializer = this@BaseClient.serializer
        }

        install(Logging) {
            level = logLevel
        }

        defaultRequest {
            header.forEach {
                parameter(it.key, it.value)
            }
            if (this.method != HttpMethod.Get) contentType(ContentType.Application.Json)
        }

    }
}

On Android works correctly on iOS it crash with this error: InvalidMutabilityException: mutation attempt of frozen

If i delete all configuration in install command and configuration of default request, it doesn't crash on iOS.

So if I change from

        install(JsonFeature) {
            serializer = this@BaseClient.serializer
        }

        install(Logging) {
            level = logLevel
        }

        defaultRequest {
            header.forEach {
                parameter(it.key, it.value)
            }
            if (this.method != HttpMethod.Get) contentType(ContentType.Application.Json)
        }

        install(JsonFeature)

        install(Logging)

it works correctly.

Is this my bug or yours?

My Environment:

val commonMain by getting {

            dependencies {
                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.9-native-mt")
                implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.0.0")
                implementation("io.ktor:ktor-client-serialization:1.4.1")
                implementation("io.ktor:ktor-client-json:1.4.1")
                implementation("io.ktor:ktor-client-serialization:1.4.1")
                implementation("io.ktor:ktor-client-core:1.4.1")
                implementation("io.ktor:ktor-client-cio:1.4.1")
                implementation("io.ktor:ktor-client-logging:1.4.1")
            }

        }
        val androidMain by getting {
            dependencies {
                implementation("io.ktor:ktor-client-android:1.4.1")
                implementation("androidx.lifecycle:lifecycle-extensions:2.2.0")
                implementation("androidx.lifecycle:lifecycle-viewmodel-ktx:2.2.0")
                implementation("io.ktor:ktor-client-serialization-jvm:1.4.1")
            }
        }
        val iosMain by getting {
            dependencies {
                implementation("io.ktor:ktor-client-ios:1.4.1")
                implementation("io.ktor:ktor-client-serialization-iosx64:1.4.1")

                implementation("com.github.florent37:multiplatform-livedata-ios:0.0.5")
            }
        }

InvalidMutabilityException: Configuration issues for ios

https://youtrack.jetbrains.com/issue/KTOR-1251

There is such a ktor configuration only for ios. Ignored ssl certificate.

    actual fun get(): HttpClient {
        return HttpClient(Ios) {
            engine {
                configureSession {
                    handleChallenge { session, task, challenge, completionHandler ->
                        val trust = challenge.protectionSpace.serverTrust
                        completionHandler(NSURLSessionAuthChallengeUseCredential, NSURLCredential.create(trust))
                    }
                }
            }
            install(JsonFeature) {
                serializer = KotlinxSerializer(
                    kotlinx.serialization.json.Json {
                        isLenient = true
                        ignoreUnknownKeys = true
                    }
                )
            }
            install(Logging) {
                logger = Logger.SIMPLE
                //level = LogLevel.ALL
            }
            install(HttpTimeout) {
                connectTimeoutMillis = TIMEOUT_MILLIS
                requestTimeoutMillis = TIMEOUT_MILLIS
            }
        }
    }
}``` 

I get an error on the platform: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.engine.ios.IosClientEngineConfig
Also tried this client configuration

```actual object HttpClientImpl {
    actual fun get(): HttpClient {
        return HttpClient {
            engine {
                Ios.config {
                    configureSession {
                        handleChallenge { session, task, challenge, completionHandler ->
                            val trust = challenge.protectionSpace.serverTrust
                            completionHandler(NSURLSessionAuthChallengeUseCredential, NSURLCredential.create(trust))
                        }
                    }
                }
            }

                    install(JsonFeature) {
                        serializer = KotlinxSerializer(
                                kotlinx.serialization.json.Json {
                                    isLenient = true
                                    ignoreUnknownKeys = true
                                }
                        )
                    }

                    install(Logging) {
                        logger = Logger.SIMPLE
                        //level = LogLevel.ALL
                    }

                    install(HttpTimeout) {
                        connectTimeoutMillis = TIMEOUT_MILLIS
                        requestTimeoutMillis = TIMEOUT_MILLIS
                    }
                }
            }
}```

Then the configuration block is ignored.

"InvalidMutabilityException: Frozen during lazy computation" when using by lazy for HttpClient

https://youtrack.jetbrains.com/issue/KTOR-1087

After updating PeopleInSpace (https://github.com/joreilly/PeopleInSpace) to following dependencies

    const val kotlinCoroutines = "1.3.9-native-mt-2"
    const val ktor = "1.4.1"
    const val kotlinxSerialization = "1.0.0-RC2"

getting following crash in iOS client (as reported in https://github.com/joreilly/PeopleInSpace/issues/11)

Uncaught Kotlin exception: kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[MainDispatcher, Continuation @ $<init>$lambda-0COROUTINE$5]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
    at 0   PeopleInSpaceSwiftUI                0x000000010e1e2b1c kfun:kotlin.Error#<init>(kotlin.String?;kotlin.Throwable?){} + 124
    at 1   PeopleInSpaceSwiftUI                0x000000010e38b83c kfun:kotlinx.coroutines.CoroutinesInternalError#<init>(kotlin.String;kotlin.Throwable){} + 124
    at 2   PeopleInSpaceSwiftUI                0x000000010e3d72e9 kfun:kotlinx.coroutines.DispatchedTask#handleFatalException(kotlin.Throwable?;kotlin.Throwable?){} + 953
    at 3   PeopleInSpaceSwiftUI                0x000000010e3d6ec7 kfun:kotlinx.coroutines.DispatchedTask#run(){} + 3543
    at 4   PeopleInSpaceSwiftUI                0x000000010e40a0b8 kfun:kotlinx.coroutines.DarwinMainDispatcher.dispatch$lambda-0#internal + 88
    at 5   PeopleInSpaceSwiftUI                0x000000010e40a48e kfun:kotlinx.coroutines.DarwinMainDispatcher.$dispatch$lambda-0$FUNCTION_REFERENCE$43.invoke#internal + 62
    at 6   PeopleInSpaceSwiftUI                0x000000010e40a4ee kfun:kotlinx.coroutines.DarwinMainDispatcher.$dispatch$lambda-0$FUNCTION_REFERENCE$43.$<bridge-UNN>invoke(){}#internal + 62
    at 7   PeopleInSpaceSwiftUI                0x000000010e40b5a7 _6f72672e6a6574627261696e732e6b6f746c696e783a6b6f746c696e782d636f726f7574696e65732d636f7265_knbridge8 + 183
    at 8   libdispatch.dylib                   0x000000010f1c28ac _dispatch_call_block_and_release + 12
    at 9   libdispatch.dylib                   0x000000010f1c3a88 _dispatch_client_callout + 8
    at 10  libdispatch.dylib                   0x000000010f1d1f23 _dispatch_main_queue_callback_4CF + 1152
    at 11  CoreFoundation                      0x00007fff203a8276 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
    at 12  CoreFoundation                      0x00007fff203a2b06 __CFRunLoopRun + 2685
    at 13  CoreFoundation                      0x00007fff203a1b9e CFRunLoopRunSpecific + 567
    at 14  GraphicsServices                    0x00007fff2b773db3 GSEventRunModal + 139
    at 15  UIKitCore                           0x00007fff24660af3 -[UIApplication _run] + 912
    at 16  UIKitCore                           0x00007fff24665a04 UIApplicationMain + 101
    at 17  PeopleInSpaceSwiftUI                0x000000010e1281cb main + 75 (/Users/mbonnin/git/PeopleInSpace/ios/PeopleInSpaceSwiftUI/PeopleInSpaceSwiftUI/AppDelegate.swift:5:7)
    at 18  libdyld.dylib                       0x00007fff20257415 start + 1
Caused by: kotlin.native.concurrent.InvalidMutabilityException: Frozen during lazy computation
    at 0   PeopleInSpaceSwiftUI                0x000000010e1ea07d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93
    at 1   PeopleInSpaceSwiftUI                0x000000010e1e2c2b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91
    at 2   PeopleInSpaceSwiftUI                0x000000010e1e2e7b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91
    at 3   PeopleInSpaceSwiftUI                0x000000010e218d6b kfun:kotlin.native.concurrent.InvalidMutabilityException#<init>(kotlin.String){} + 91
    at 4   PeopleInSpaceSwiftUI                0x000000010e21b419 kfun:kotlin.native.concurrent.FreezeAwareLazyImpl.getOrInit#internal + 1641
    at 5   PeopleInSpaceSwiftUI                0x000000010e21b95e kfun:kotlin.native.concurrent.FreezeAwareLazyImpl#<get-value>(){}1:0 + 670
    at 6   PeopleInSpaceSwiftUI                0x000000010e13f183 kfun:com.surrus.common.remote.PeopleInSpaceApi.<get-client>#internal + 323
    at 7   PeopleInSpaceSwiftUI                0x000000010e13feb7 kfun:com.surrus.common.remote.PeopleInSpaceApi.$fetchPeopleCOROUTINE$0#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 2983
    at 8   PeopleInSpaceSwiftUI                0x000000010e141a74 kfun:com.surrus.common.remote.PeopleInSpaceApi#fetchPeople(){}com.surrus.common.remote.AstroResult + 244
    at 9   PeopleInSpaceSwiftUI                0x000000010e146666 kfun:com.surrus.common.repository.PeopleInSpaceRepository.$fetchAndStorePeopleCOROUTINE$2.invokeSuspend#internal + 886
    at 10  PeopleInSpaceSwiftUI                0x000000010e146bb4 kfun:com.surrus.common.repository.PeopleInSpaceRepository.fetchAndStorePeople#internal + 244
    at 11  PeopleInSpaceSwiftUI                0x000000010e1487da kfun:com.surrus.common.repository.PeopleInSpaceRepository.$<init>$lambda-0COROUTINE$5.invokeSuspend#internal + 602
    at 12  PeopleInSpaceSwiftUI                0x000000010e20bba6 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 758
    at 13  PeopleInSpaceSwiftUI                0x000000010e3d6be2 kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2802
    at 14  PeopleInSpaceSwiftUI                0x000000010e40a0b8 kfun:kotlinx.coroutines.DarwinMainDispatcher.dispatch$lambda-0#internal + 88
    ... and 14 more common stack frames skipped
CoreSimulator 732.17 - Device: iPhone 11 (8BACA2EC-79DF-49D0-937D-AFA747B52692) - Runtime: iOS 14.0 (18A372) - DeviceType: iPhone 11
(lldb)

The particular code that's triggering this is in PeopleInSpaceRepository. That uses Ktor to make api request and then stores in db (but see same issue if I only do ktor request there)

    init {
        GlobalScope.launch(Dispatchers.Main) {
            fetchAndStorePeople()
        }
    }

Ktor OkHttp client uses blocking calls inside Coroutine Dispatcher (ktor-okhttp-dispatcher) with default maxPoolSize = 4.

https://youtrack.jetbrains.com/issue/KTOR-1226

Ktor OkHttp client uses blocking calls inside Coroutine Dispatcher (ktor-okhttp-dispatcher) with default maxPoolSize = 4.
I guess the default HttpClientEngineConfig.threadsCount for OkHttp should be increased.

java.net.SocketInputStream.socketRead0(SocketInputStream.java)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at okio.InputStreamSource.read(JvmOkio.kt:90)
	at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:129)
	at okio.RealBufferedSource.read(RealBufferedSource.kt:188)
	at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read(Http1ExchangeCodec.kt:331)
	at okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.kt:412)
	at okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.kt:276)
	at okio.RealBufferedSource.read(RealBufferedSource.kt:82)
	at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$invokeSuspend$$inlined$use$lambda$1.invoke(OkHttpEngine.kt:154)
	at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$invokeSuspend$$inlined$use$lambda$1.invoke(OkHttpEngine.kt)
	at io.ktor.utils.io.ByteBufferChannel.writeAvailable(ByteBufferChannel.kt:1593)
	at io.ktor.utils.io.ByteBufferChannel.write$suspendImpl(ByteBufferChannel.kt:1620)
	at io.ktor.utils.io.ByteBufferChannel.write(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteWriteChannel$DefaultImpls.write$default(ByteWriteChannel.kt:100)
	at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invokeSuspend(OkHttpEngine.kt:152)
	at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invoke(OkHttpEngine.kt)
	at io.ktor.utils.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:129)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

OkHTTP client engine tries to close the connection twice during the closing handshake

https://youtrack.jetbrains.com/issue/KTOR-1374

Here's how I noticed:

When I cancel the coroutine containing the WebSocket connection, KTor closes the connection with a 1011. If the server then responds by closing the connection without a status code, KTor will get a 1005 and it'll try to pass it to OkHTTP.

However, 1005 is a reserved status, so OkHTTP refuses it when Ktor passes it in the following lines: https://github.com/ktorio/ktor/blob/6053fa44a61fdfc5474ff909c0224e56877b5da1/ktor-client/ktor-client-okhttp/jvm/src/io/ktor/client/engine/okhttp/OkHttpWebsocketSession.kt#L62

Here's an example of a traceback you'd get:

    java.util.concurrent.CancellationException: ActorCoroutine was cancelled
        at kotlinx.coroutines.ExceptionsKt.CancellationException(Exceptions.kt:22)
        at kotlinx.coroutines.channels.ActorCoroutine.onCancelling(Actor.kt:134)
        at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:332)
        at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:916)
        at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:875)
        at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:840)
        at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:111)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
     Caused by: java.lang.IllegalArgumentException: Code 1005 is reserved and may not be used.
        at okhttp3.internal.ws.WebSocketProtocol.validateCloseCode(WebSocketProtocol.kt:134)
        at okhttp3.internal.ws.RealWebSocket.close(RealWebSocket.kt:435)
        at okhttp3.internal.ws.RealWebSocket.close(RealWebSocket.kt:427)
        at io.ktor.client.engine.okhttp.OkHttpWebsocketSession$outgoing$1.invokeSuspend(OkHttpWebsocketSession.kt:62)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) 
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) 
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738) 
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) 
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

I think there are technically two different issues happening at the same time:

Ktor is trying to close the connection twice. The first time with a 1011 status (which I can confirm in the server logs) and the second time with a 1005 (because the server didn't provide a closing status code).
Ktor is trying to close the connection with a reserved status code.

"Unfinished workers detected" using client on native

ResponseException is no longer serializable starting from 1.4.0 (breaking change)

https://youtrack.jetbrains.com/issue/KTOR-1552

Stack trace:

io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
java.io.NotSerializableException: io.ktor.utils.io.concurrent.SharedJvmKt$threadLocal$1
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)
	at ExceptionsTest.serialize(ExceptionsTest.kt:26)
	at ExceptionsTest.testResponseExceptionSerializable(ExceptionsTest.kt:17)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)

Here is a test to reproduce the issue:

import io.ktor.client.call.*
import io.ktor.client.features.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.util.date.*
import io.ktor.utils.io.*
import java.io.*
import kotlin.coroutines.*
import kotlin.test.*

class ExceptionsTest {

    @Test
    fun testResponseExceptionSerializable() {
        val exception = createResponseException()

        val serialized = serialize(exception)
        val deserialized = deserialize(serialized)

        deserialized as ResponseException
    }

    private fun serialize(obj: Any): ByteArray {
        val b = ByteArrayOutputStream()
        val o = ObjectOutputStream(b)
        o.writeObject(obj)
        return b.toByteArray()
    }

    private fun deserialize(bytes: ByteArray): Any? {
        val b = ByteArrayInputStream(bytes)
        val o = ObjectInputStream(b)
        return o.readObject()
    }
}

private fun createResponseException(): ResponseException = ResponseException(object : HttpResponse() {
    override val call: HttpClientCall
        get() = TODO("Not yet implemented")
    override val status: HttpStatusCode
        get() = TODO("Not yet implemented")
    override val version: HttpProtocolVersion
        get() = TODO("Not yet implemented")
    override val requestTime: GMTDate
        get() = TODO("Not yet implemented")
    override val responseTime: GMTDate
        get() = TODO("Not yet implemented")
    override val content: ByteReadChannel
        get() = TODO("Not yet implemented")
    override val headers: Headers
        get() = TODO("Not yet implemented")
    override val coroutineContext: CoroutineContext
        get() = TODO("Not yet implemented")

    override fun toString(): String = "FakeCall"
}, cachedResponseText = "Fake text")

Make curl engine option sslVerify public

Client: NPE in FormDataContentKt -> Input.copyTo

https://youtrack.jetbrains.com/issue/KTOR-1349

I've received the following exception report for an Android app.
It was sporadic and I cannot reproduce the problem. The problem seems to lie within Ktor itself so I'll simply share what I can.

FATAL EXCEPTION: k.a0 Dispatcher = okhttp3.OkHttpClient Dispatcher
java.lang.NullPointerException
	at h.b.a.g.j.b$a.l(Unknown Source:58) = io.ktor.client.request.forms.FormDataContentKt$copyTo$2.invokeSuspend(java.lang.Object)
	at kotlin.c0.k.a.a.n(Unknown Source:9) = kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(java.lang.Object)
	at kotlinx.coroutines.a1.run(Unknown Source:93)
	at kotlinx.coroutines.d3.b.b0(Unknown Source:0)
	at kotlinx.coroutines.d3.b$a.c(Unknown Source:14)
	at kotlinx.coroutines.d3.b$a.m(Unknown Source:28)
	at kotlinx.coroutines.d3.b$a.run(Unknown Source:0)

This is the relevant mapping:

io.ktor.client.request.forms.FormDataContentKt$copyTo$2 -> h.b.a.g.j.b$a:
    io.ktor.utils.io.WriterSuspendSession p$ -> p
    java.lang.Object L$0 -> q
    io.ktor.utils.io.core.Input $this_copyTo -> s
    int label -> r
    java.lang.Object invoke(java.lang.Object,java.lang.Object) -> M
    kotlin.coroutines.Continuation create(java.lang.Object,kotlin.coroutines.Continuation) -> d
    java.lang.Object invokeSuspend(java.lang.Object) -> l

The relevant code from Ktor:
https://github.com/ktorio/ktor/blob/a6d68420ee0b272cd09d235057427a6601736487/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L154

private suspend fun Input.copyTo(channel: ByteWriteChannel) {
    if (this is ByteReadPacket) {
        channel.writePacket(this)
        return
    }

    channel.writeSuspendSession {
        while (!this@copyTo.endOfInput) {
            tryAwait(1)
            val buffer = request(1)!!                                   // <-- likely cause
            val size = this@copyTo.readAvailable(buffer)

            if (size < 0) continue
            written(size)
        }
    }
}

CIO: client engine exceptions are both logged and thrown

https://youtrack.jetbrains.com/issue/KTOR-1127

In the CIO client engine exceptions like SocketTimeoutException are logged to standard error by the coroutines' uncaught exception handler and thrown by client.get().

import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*

suspend fun main(): Unit = withContext(Dispatchers.Default) {
    HttpClient(CIO) {
        install(HttpTimeout) {
            socketTimeoutMillis = 5
        }
    }.use { client ->
        try {
            client.get<HttpResponse>("http://www.google.com").readText()
        } catch (e: Throwable) {
            println("Caught: $e")
        }
    }
}

Output:

Exception in thread "kotlinx.coroutines.DefaultExecutor" java.net.SocketTimeoutException
	at io.ktor.network.util.UtilsKt$withSocketTimeout$2.invokeSuspend(Utils.kt:23)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:32)
	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:113)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:188)
	at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:122)
	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
	at kotlinx.coroutines.CancellableContinuationImpl.cancel(CancellableContinuationImpl.kt:177)
	at kotlinx.coroutines.CancellableContinuationImpl.parentCancelled$kotlinx_coroutines_core(CancellableContinuationImpl.kt:184)
	at kotlinx.coroutines.ChildContinuation.invoke(JobSupport.kt:1484)
	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1511)
	at kotlinx.coroutines.JobSupport.tryMakeCancelling(JobSupport.kt:792)
	at kotlinx.coroutines.JobSupport.makeCancelling(JobSupport.kt:752)
	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:668)
	at kotlinx.coroutines.JobSupport.cancelCoroutine(JobSupport.kt:655)
	at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:128)
	at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:497)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
	at java.base/java.lang.Thread.run(Thread.java:832)

Caught: io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=http://www.google.com/, socket_timeout=5] ms

HttpRedirect feature alters Location header value

CIO Engine's HttpClient may fail when trying to send large size binary data.

https://youtrack.jetbrains.com/issue/KTOR-1247

Ktor Version and Engine Used (client or server and name)

Client CIO, 1.4.1

Describe the bug

This behavior is not reproduced in the case of HttpClient with Apache Engine.

To Reproduce

Sample code is below.

@KtorExperimentalAPI
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    routing {
        post("/multipart") {
            val multipartData = call.receiveMultipart()
            multipartData.forEachPart {
                if (it is PartData.FileItem) {
                    println(
                        withContext(Dispatchers.IO) {
                            it.streamProvider().readAllBytes().size
                        }
                    )
                }
            }
            call.respondText("test")
        }
        get("/execute/cio") {
            repeat(10) {
                request(HttpClient(CIO))
            }
            call.respondText("success")
        }
        get("/execute/apache") {
            repeat(10) {
                request(HttpClient(Apache))
            }
            call.respondText("success")
        }
    }
}

suspend fun request(client: HttpClient) {
    val body = formData {
        val headers = headersOf(
            HttpHeaders.ContentDisposition,
            ContentDisposition.File
                .withParameter(ContentDisposition.Parameters.Name, "file")
                .withParameter(ContentDisposition.Parameters.FileName, "test.jpg")
                .toString()
        )
        appendInput("file", headers) {
            // dummy 10MB binary file
            ByteArray(10_000_000).inputStream().asInput()
        }
    }
    client.use {
        it.submitFormWithBinaryData<Unit>(
            url = "http://localhost:8080/multipart",
            formData = body
        )
    }
}

(full code is in https://github.com/ma2Cta/ktor-multipart-request-test/blob/main/src/Application.kt )

Start the server.
Run $ curl http://localhost:8080/execute/apache a few times.

10 times 10000000 will be printed as expected.

Run $ curl http://localhost:8080/execute/cio a few times.

You won't get 10000000 10 times, and you'll probably get a number less than 10000000 the first few times

Expected

The correct request is sent even in the case of CIO Engine.

Timeout feature: android engine throws Java's SocketTimeoutException instead of ConnectTimeoutException

Apache client engine sometimes hits an unrecoverable socket timeout when using ChannelWriterContent

https://youtrack.jetbrains.com/issue/KTOR-1149

Test to reproduce below. Tested on two different machines (Windows and macOS), works fine with CIO. With 500 test iterations, happens at least once most times.

import io.ktor.application.*
import io.ktor.client.*
import io.ktor.client.engine.apache.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.http.*
import io.ktor.http.content.*
import io.ktor.request.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.cio.*
import io.ktor.server.engine.*
import io.ktor.utils.io.*
import kotlinx.coroutines.runBlocking
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.RepeatedTest
import java.util.concurrent.TimeUnit

class KtorApacheBugTest {

    private val port = 8000
    private val server = embeddedServer(
        factory = CIO,
        port = port,
    ) {
        routing {
            put("/") {
                call.receiveChannel().discard()
                call.respond(HttpStatusCode.OK)
            }
        }
    }
    private val client = HttpClient(Apache) {
        install(HttpTimeout) {
            socketTimeoutMillis = 10_000
            connectTimeoutMillis = 10_000
        }
    }

    @BeforeEach
    fun setUp() {
        server.start()
    }

    @AfterEach
    fun tearDown() {
        client.close()
        server.stop(0, 1, TimeUnit.SECONDS)
    }

    @RepeatedTest(500)
    fun `eventually hits a socket timeout`(): Unit = runBlocking {
        val chunk = (0 until 10_000).map { it.toByte() }.toByteArray()

        client.put<String>("http://localhost:${port}") {
            body = ChannelWriterContent(
                body = {
                    writeFully(chunk)
                },
                contentType = null,
            )
        }
    }
}

Another simple test (seems the issue is with ChannelWriterContent) below.

This works:

@RepeatedTest(500)
    fun `does not hit a socket timeout`(): Unit = runBlocking {
        client.put<String>("http://localhost:${port}") {
            body = LocalFileContent(File("gradle/wrapper/gradle-wrapper.jar"))
        }
    }

This does not:

@RepeatedTest(500)
    fun `eventually hits a socket timeout`(): Unit = runBlocking {
        val chunk = File("gradle/wrapper/gradle-wrapper.jar").readBytes()

        client.put<String>("http://localhost:${port}") {
            body = ChannelWriterContent(
                body = {
                    writeFully(chunk)
                },
                contentType = null,
            )
        }
    }

Sending the same via ByteArrayContent works fine as well.

Ktor 1.4.1 fails to upload file (submit form) when logging level = LogLevel.BODY

Overload resolution ambiguity: HttpClient.ws

Content-type header is missing in Request when using LogLevel.BODY

Jetty client engine with high concurrency randomly freezes, throws NPEs, ISEs or EofExceptions

https://youtrack.jetbrains.com/issue/KTOR-1120

Exception in thread "main" java.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at org.eclipse.jetty.io.SelectorManager.connect(SelectorManager.java:193)
	at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:409)
	at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:382)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt:61)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt)
	at io.ktor.client.engine.jetty.UtilsKt.withPromise(utils.kt:14)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.connect(JettyHttpRequest.kt:59)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.executeRequest(JettyHttpRequest.kt:31)
	at io.ktor.client.engine.jetty.JettyHttp2Engine.execute(JettyHttp2Engine.kt:38)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more
	Suppressed: java.lang.NullPointerException
		... 17 more

import io.ktor.client.*
import io.ktor.client.engine.jetty.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*

suspend fun main() = withContext(Dispatchers.Default) {
   HttpClient(Jetty) {
      expectSuccess = false

      install(HttpTimeout) {
         connectTimeoutMillis = 2_000
         socketTimeoutMillis = 2_000
      }

      engine {
         threadsCount = 200
      }
   }.use { client ->
      coroutineScope {
         repeat(5) {
            repeat(200) {
               launch {
                  client.get<HttpResponse>("https://esi.evetech.net/latest/markets/10000040/orders/?page=1")
                     .readText()
                     .let { println(it.length) }
               }
            }
         }
      }
   }

   println()
   println("Done")
}

Native: ResponseObserver hangs coroutine indefinitely

CertificatePinner buildErrorMessage outputs CertificatesInfo reference and variable name (HASH_ALGORITHM_SHA_256) instead of value

CIO: client throws ConnectTimeoutException when endpoint.connectRetryAttempts = 0

Client: InvalidCacheStateException when Vary header differs for 200 and 304 responses

"FreezingException: freezing of InvokeOnCompletion has failed" using native-mt coroutines

https://youtrack.jetbrains.com/issue/KTOR-973

Hi.
I have a problem on Ktor client native.
I’ve been struggling to make the code below works after migrating kotlin to 1.4.0 version
while migrating, I changed coroutine version to 1.3.9-native-mt

class TestFreezingViewModel(val client: HttpClient) {
    fun call() {
         CoroutineScope(Dispatchers.Main).launch(start = CoroutineStart.LAZY) {
            try {
                client.get<String>("https://test.com") {}
            } catch (e: Throwable) {
                e.printStackTrace()
            }
        }.start()
    }
}

swift side code

TestFreezingViewModel().call()

while the code running, The error below occurs

kotlin.native.concurrent.FreezingException: freezing of InvokeOnCompletion[InvokeOnCompletion@1b76e08] has failed, first blocker is HttpClient[io.ktor.client.engine.ios.IosClientEngine@1b1b748]
    at 0   kotlinIOS                           0x000000010bf93fed kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93
    at 1   kotlinIOS                           0x000000010bf8cb9b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91
    at 2   kotlinIOS                           0x000000010bf8cdeb kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91
    at 3   kotlinIOS                           0x000000010bfc2c13 kfun:kotlin.native.concurrent.FreezingException#<init>(kotlin.Any;kotlin.Any){} + 643
    at 4   kotlinIOS                           0x000000010bfc4297 ThrowFreezingException + 231
    at 5   kotlinIOS                           0x000000010c0b2f4f FreezeSubgraph + 2815
    at 6   kotlinIOS                           0x000000010c0cc17b Kotlin_Worker_freezeInternal + 27
    at 7   kotlinIOS                           0x000000010bfc2d3e kfun:kotlin.native.concurrent#freeze@0:0(){0§<kotlin.Any?>}0:0 + 62
    at 8   kotlinIOS                           0x000000010c10c749 kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 1881
    at 9   kotlinIOS                           0x000000010c10bf76 kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 198
    at 10  kotlinIOS                           0x000000010c33b15b kfun:io.ktor.client.engine#createCallContext@io.ktor.client.engine.HttpClientEngine(kotlinx.coroutines.Job){}kotlin.coroutines.CoroutineContext + 1451
    at 11  kotlinIOS                           0x000000010c302d08 kfun:io.ktor.client.engine.HttpClientEngine.$executeWithinCallContextCOROUTINE$9.invokeSuspend#internal + 1048
    at 12  kotlinIOS                           0x000000010c303444 kfun:io.ktor.client.engine.HttpClientEngine.executeWithinCallContext#internal + 308
    at 13  kotlinIOS                           0x000000010c30420a kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invokeSuspend#internal + 1562
    at 14  kotlinIOS                           0x000000010c304bf8 kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invoke#internal + 312
    at 15  kotlinIOS                           0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
    at 16  kotlinIOS                           0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395
    at 17  kotlinIOS                           0x000000010c29d50a kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal + 474
    at 18  kotlinIOS                           0x000000010c295759 kfun:io.ktor.util.pipeline.Pipeline#execute(1:1;1:0){}1:0 + 361
    at 19  kotlinIOS                           0x000000010c326be5 kfun:io.ktor.client.features.HttpSend.DefaultSender.$executeCOROUTINE$27.invokeSuspend#internal + 1365
    at 20  kotlinIOS                           0x000000010c327244 kfun:io.ktor.client.features.HttpSend.DefaultSender.execute#internal + 308
    at 21  kotlinIOS                           0x000000010c324fbc kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invokeSuspend#internal + 2236
    at 22  kotlinIOS                           0x000000010c3260f8 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invoke#internal + 312
    at 23  kotlinIOS                           0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
    at 24  kotlinIOS                           0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395
    at 25  kotlinIOS                           0x000000010c29d2a5 kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceedWith#internal + 213
    at 26  kotlinIOS                           0x000000010c3173eb kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-0COROUTINE$17.invokeSuspend#internal + 907
    at 27  kotlinIOS                           0x000000010c317c58 kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-0COROUTINE$17.invoke#internal + 312
    at 28  kotlinIOS                           0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
    at 29  kotlinIOS                           0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395

when Coroutine’s invokeOnCompletion is invoked, freeze() is called.
and it tells that HttpClient can’t be frozen
When I checked the source code
HttpClient.init calls preventFreeze()
I would like to know the intention of preventFreeze and what is the proper approach to use on developer’s side.
please check if this is bug, other wise, kindly let me share some advice.
Thanks

CIO: java.io.EOFException when server requires client certificate and restricts the authority

	reading	remainingBefore	remainingAfter	content	position
before		281	281
reading authorizationSize	2	281	279	279
reading size	2	279	277	91
reading CA 1	91	277	186	CA...	91 (should 93)
reading size	2	186	184	91
reading CA 2	91	184	93	CA	182 (should 186)
reading size	2	93	91	91
reading CA 3	91	91	0	CA	273 (should 279)

Calling HttpStatement#toString more than once throws IllegalArgumentException

Implement runtime check of using `native-mt` coroutines

Ktor websocket client passes configured max frame as timeout millis

1.4.0: breaking change by making response nullable in ResponseException

Parser Exception in header with character code 1 not allowed

https://youtrack.jetbrains.com/issue/KTOR-860

The following URL

https://data.smartdublin.ie/cgi-bin/rtpi

returns valid results with other HTTP clients, including IntelliJ HTTP Requests. However with Ktor, independently of engine, it throws a parser exception (side note, it would be good to indicate the header in the exception result)

Exception in thread "main" io.ktor.http.cio.ParserException: Character with code 1 is not allowed in header names, 
HTTP/1.1 200 OKServer: nginx/1.17.8Date: Sun, 19 Jul 2020 07:12:42 GMTContent-Type: application/json; charset=utf-8Content-Length: 1596054Connection: keep-aliveX-Powered-By: ASP.NETSet-Cookie: visid_incap_1822411=6ycr4UzsSv+6sdm+QnNmpmnyE18AAAAAQUIPAAAAAACm9JYEnYu4a0fTA7VxKyxu; expires=Sun, 18 Jul 2021 08:40:35 GMT; HttpOnly; path=/; Domain=.nationaltransport.ieSet-Cookie: incap_ses_535_1822411=vjDoPwoqvTdguklauLNsB2nyE18AAAAAoieq1zxaD46bWhzEJyQd3w==; path=/; Domain=.nationaltransport.ieSet-Cookie: ___utmvmzauvysSB=WQYQExBjiZv; path=/; Max-Age=900Set-Cookie: ___utmvazauvysSB=TjhnJRz; path=/; Max-Age=900
	at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:279)
	at io.ktor.http.cio.HttpParserKt.parseHeaderValue(HttpParser.kt:261)
	at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:119)
	at io.ktor.http.cio.HttpParserKt.parseResponse(HttpParser.kt:73)
	at io.ktor.http.cio.HttpParserKt$parseResponse$1.invokeSuspend(HttpParser.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Process finished with exit code 1

To reproduce see attached code.

Native Client call freezes the CoroutineExceptionHandler of the coroutine it's ran from

https://youtrack.jetbrains.com/issue/KTOR-1073

(This issue is about Ktor 1.4.1 but I can't select it as maybe it's too early)

I have an iOS app which has been using the Ktor Client for a long time now.
As coroutines and Kotlin/Native in general isn't (weren't ?) good in multithread, this app uses a single thread.

Ktor 1.4.0 now depends on the native-mt version of coroutines, which is great, but this comes with problems.
The Ktor client mostly didn't work on Kotlin Native due to issues which have been fixed in 1.4.1.
Now, the only issue left (before the next one) preventing update from Ktor 1.3.x to 1.4.x is that Ktor 1.4 freezes the coroutine it's ran in.

I don't know why it does that and maybe there's a good reason, but I don't know how to work around that.

So the issue is, maybe there's a lack of documentation, or maybe it shouldn't freeze the coroutine in the first place unless it's needed (but I thought that kotlinx-coroutines freezes coroutines switching thread already, so I don't understand why Ktor would do anything more than that), but right now I don't understand what I'm supposed to do to make this work. I'm attaching a small project to illustrate.

How do you make the attached project work without removing any ensureNeverFrozen() and without dramatically changing the structure? I tried wrapping the call to HttpClient.get in various ways (including withContext(MainScope().coroutineContext)) but was not successful.

Here is the reproducer project: https://github.com/Ribesg/kotlin-reproducer/tree/ktor-1073
To reproduce run xcode.app from the Xcode.

kotlin.native.concurrent.FreezingException: freezing of InvokeOnCompletion[InvokeOnCompletion@28fd2c8] has failed, first blocker is com.example.app.feature.startup.Startup@33edbf8
    at 0   xcode                               0x000000010a58ee8d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
    at 1   xcode                               0x000000010a587d7b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
    at 2   xcode                               0x000000010a587f3b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
    at 3   xcode                               0x000000010a5bd423 kfun:kotlin.native.concurrent.FreezingException#<init>(kotlin.Any;kotlin.Any){} + 643 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:15:9)
    at 4   xcode                               0x000000010a5beaa7 ThrowFreezingException + 231 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:87:15)
    at 5   xcode                               0x000000010a6aaf0f FreezeSubgraph + 2815
    at 6   xcode                               0x000000010a6c49bb Kotlin_Worker_freezeInternal + 27
    at 7   xcode                               0x000000010a5bd54e kfun:kotlin.native.concurrent#freeze@0:0(){0§<kotlin.Any?>}0:0 + 62 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:33:5)
    at 8   xcode                               0x000000010a702231 kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 1921 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:468:54)
    at 9   xcode                               0x000000010a701a2f kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 207 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:452:9)
    at 10  xcode                               0x000000010a9320ff kfun:io.ktor.client.engine#createCallContext@io.ktor.client.engine.HttpClientEngine(kotlinx.coroutines.Job){}kotlin.coroutines.CoroutineContext + 1487 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/posix/src/io/ktor/client/engine/HttpClientEngineNative.kt:26:9)
    at 11  xcode                               0x000000010a8f71af kfun:io.ktor.client.engine.HttpClientEngine.$executeWithinCallContextCOROUTINE$9.invokeSuspend#internal + 1087 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/engine/HttpClientEngine.kt:77:27)
    at 12  xcode                               0x000000010a8f78f0 kfun:io.ktor.client.engine.HttpClientEngine.executeWithinCallContext#internal + 320 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/engine/HttpClientEngine.kt:76:21)
    at 13  xcode                               0x000000010a8f870a kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invokeSuspend#internal + 1594 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/engine/HttpClientEngine.kt:66:32)
    at 14  xcode                               0x000000010a8f9127 kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invoke#internal + 327 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/engine/HttpClientEngine.kt:57:64)
    at 15  xcode                               0x000000010a890187 kfun:io.ktor.util.pipeline.SuspendFunctionGun.$loop$<anonymous>_2COROUTINE$7.invokeSuspend#internal + 695 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:17:6)
    at 16  xcode                               0x000000010a8906e7 kfun:io.ktor.util.pipeline.SuspendFunctionGun.$loop$<anonymous>_2COROUTINE$7.invoke#internal + 215 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:17:4)
    at 17  xcode                               0x000000010a88e8f5 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1253 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:157:99)
    at 18  xcode                               0x000000010a88decd kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceed(){}1:0 + 413 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:113:13)
    at 19  xcode                               0x000000010a88e33c kfun:io.ktor.util.pipeline.SuspendFunctionGun#execute(1:0){}1:0 + 492 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:133:16)
    at 20  xcode                               0x000000010a8866a8 kfun:io.ktor.util.pipeline.Pipeline#execute(1:1;1:0){}1:0 + 376 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/Pipeline.kt:77:41)
    at 21  xcode                               0x000000010a91ac2c kfun:io.ktor.client.features.HttpSend.DefaultSender.$executeCOROUTINE$27.invokeSuspend#internal + 1404 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:128:50)
    at 22  xcode                               0x000000010a91b270 kfun:io.ktor.client.features.HttpSend.DefaultSender.execute#internal + 320 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:120:26)
    at 23  xcode                               0x000000010a918fb1 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invokeSuspend#internal + 2273 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:92:42)
    at 24  xcode                               0x000000010a91a0f7 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invoke#internal + 327 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:82:71)
    at 25  xcode                               0x000000010a890187 kfun:io.ktor.util.pipeline.SuspendFunctionGun.$loop$<anonymous>_2COROUTINE$7.invokeSuspend#internal + 695 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:17:6)
    at 26  xcode                               0x000000010a8906e7 kfun:io.ktor.util.pipeline.SuspendFunctionGun.$loop$<anonymous>_2COROUTINE$7.invoke#internal + 215 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:17:4)
    at 27  xcode                               0x000000010a88e8f5 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1253 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:157:99)
    at 28  xcode                               0x000000010a88decd kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceed(){}1:0 + 413 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:113:13)
    at 29  xcode                               0x000000010a88e0ce kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceedWith(1:0){}1:0 + 222 (/Users/Aleksei.Tirman/projects/jb/ktor/ktor-utils/common/src/io/ktor/util/pipeline/SuspendFunctionGun.kt:123:16)

Android: application crashes unexpectedly when making HTTP request

ktor-client-apache: thread stuck in ByteBufferChannel.writeAvailable

100% CPU usage on "Ktor-apache-client" threads after upgrading from 1.3.2 to 1.4.0

https://youtrack.jetbrains.com/issue/KTOR-1082

After updating from Ktor 1.3.2 to 1.4.0, I've noticed that some services suddenly have 100% cpu usage on some threads. It even lead to one service being completely unresponsive, as it seems like enough threads got into this broken status. Unfortunately, I was not able to get a jstack before the instance restarted, however I did get one last week, when only 1 thread was in this status. The thread dump is attached.

"Ktor-client-apache" #35 daemon prio=5 os_prio=0 tid=0x00007f995c06b000 nid=0x2a runnable [0x00007f99b62f0000]
   java.lang.Thread.State: RUNNABLE
	at io.ktor.utils.io.ByteWriteChannel$DefaultImpls.writeAvailable$default(ByteWriteChannel.kt:85)
	at io.ktor.client.engine.apache.ApacheResponseConsumer.consumeContent(ApacheResponseConsumer.kt:50)
	at org.apache.http.impl.nio.client.MainClientExec.consumeContent(MainClientExec.java:329)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.consumeContent(DefaultClientExchangeHandlerImpl.java:157)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:336)
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:121)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None

Also attached is a screenshot of the cpu usage. It looks like it happens to one thread at a time, going up to 100% usage (this screenshot doesn't correspond to the attached dump, it was a different occurrence)

I will continue to monitor this and will be able to get more information (any suggestions?) when this happens again.

IncorrectDereferenceException thrown when ktor request call from other thread

Native: InvalidMutabilityException creating HttpClient

https://youtrack.jetbrains.com/issue/KTOR-915

As of v.1.4.0, ktor client is not working on Kotlin/native iOS due to an InvalidMutabilityException.

I tested with both coroutines "1.3.9" as well as "1.3.9-native-mt", each produce the same error:

Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlinx.coroutines.ChildHandleNode@3873b88
    at 0   jgosdk                              0x000000010b7376fd kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
    at 1   jgosdk                              0x000000010b73030b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
    at 2   jgosdk                              0x000000010b73055b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
    at 3   jgosdk                              0x000000010b767deb kfun:kotlin.native.concurrent.InvalidMutabilityException#<init>(kotlin.String){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:22:60)
    at 4   jgosdk                              0x000000010b7682c2 ThrowInvalidMutabilityException + 690 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:92:11)
    at 5   jgosdk                              0x000000010b85e9ec MutationCheck + 108
    at 6   jgosdk                              0x000000010b9017b6 kfun:kotlinx.coroutines.internal.LinkedListNode#<set-_next>(kotlinx.coroutines.internal.LinkedListNode){} + 102 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:19:28)
    at 7   jgosdk                              0x000000010b901d9b kfun:kotlinx.coroutines.internal.LinkedListNode#addLast(kotlinx.coroutines.internal.LinkedListNode){} + 283 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:31:9)
    at 8   jgosdk                              0x000000010b9020e6 kfun:kotlinx.coroutines.internal.LinkedListNode#addOneIfEmpty(kotlinx.coroutines.internal.LinkedListNode){}kotlin.Boolean + 230 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:47:9)
    at 9   jgosdk                              0x000000010b8a3cbf kfun:kotlinx.coroutines.JobSupport.promoteSingleToNodeList#internal + 463 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:532:15)
    at 10  jgosdk                              0x000000010b8a291d kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 2061 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:470:25)
    at 11  jgosdk                              0x000000010b8a208f kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 207 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:449:9)
    at 12  jgosdk                              0x000000010ba59d04 kfun:io.ktor.client#HttpClient(io.ktor.client.engine.HttpClientEngineFactory<0:0>;kotlin.Function1<io.ktor.client.HttpClientConfig<0:0>,kotlin.Unit>){0§<io.ktor.client.engine.HttpClientEngineConfig>}io.ktor.client.HttpClient + 1156 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:45:36)

Circular reference for SocketException and StackOverflowError when using SLF4J logger

https://youtrack.jetbrains.com/issue/KTOR-1080

I upgraded my project to ktor 1.4.1 today.
I found an issue when I ran my unit test.
My logger failed to log the SocketTimeoutException.

Sample code:

fun main() {
    val httpClient = HttpClient(Apache) {
        install(HttpTimeout) {
            socketTimeoutMillis = 3000
        }
    }

    runBlocking {
        try {
            val get = httpClient.get<String>("https://httpbin.org/delay/10")
        } catch (e: Exception) {
            e.printStackTrace()
            LoggerFactory.getLogger("main").error("test", e)
        }
    }
}

Output:

io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=https://httpbin.org/delay/10, socket_timeout=3000] ms
	at io.ktor.client.features.HttpTimeoutKt.SocketTimeoutException(HttpTimeout.kt:177)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt.mapCause(ApacheHttpRequest.kt:57)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt.sendRequest(ApacheHttpRequest.kt:49)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt$sendRequest$1.invokeSuspend(ApacheHttpRequest.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=https://httpbin.org/delay/10, socket_timeout=3000] ms
	at io.ktor.client.features.HttpTimeoutKt.SocketTimeoutException(HttpTimeout.kt:177)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt.mapCause(ApacheHttpRequest.kt:57)
	at io.ktor.client.engine.apache.ApacheResponseConsumer.failed(ApacheResponseConsumer.kt:73)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:99)
	at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.base/java.lang.Thread.run(Thread.java:834)
	[CIRCULAR REFERENCE:io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=https://httpbin.org/delay/10, socket_timeout=3000] ms]
Caused by: java.net.SocketTimeoutException: 3,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
	... 11 more
Exception in thread "main" java.lang.StackOverflowError
	at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:590)
	at java.base/java.lang.reflect.Method.invoke(Method.java:558)
	at ch.qos.logback.classic.spi.ThrowableProxy.<init>(ThrowableProxy.java:66)
	at ch.qos.logback.classic.spi.ThrowableProxy.<init>(ThrowableProxy.java:60)
	at ch.qos.logback.classic.spi.ThrowableProxy.<init>(ThrowableProxy.java:60)
	at ch.qos.logback.classic.spi.ThrowableProxy.<init>(ThrowableProxy.java:72)

Core

Gradle 5.6.4: Duplicate class kotlinx.coroutines.*

Significant performance penalty with socket write() and file read() on JVM

Input.readTextExactBytes(n) on empty input different behavior per platform

ktor-io: JVM shared function decrease performance starting from 1.4.0

Unable to catch socket exceptions

https://youtrack.jetbrains.com/issue/KTOR-1166

Exceptions thrown from the UDP sockets cannot be caught.

Here is a test that currently fails:

import io.ktor.network.selector.*
import io.ktor.network.sockets.*
import io.ktor.util.network.*
import io.ktor.utils.io.core.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import kotlin.io.use
import kotlin.test.*

class SocketExceptionIssue : CoroutineScope {
    private val testJob = Job()
    private val selector = ActorSelectorManager(Dispatchers.Default + testJob)

    override val coroutineContext: CoroutineContext
        get() = testJob

    @AfterTest
    fun tearDown() {
        testJob.cancel()
        selector.close()
    }

    @Test
    fun testBroadcastFails(): Unit = runBlocking {
        withTimeout(15000) {
            // The following code in the assertFails lambda should throw a SocketException.
            assertFails {
                aSocket(selector)
                    .udp()
                    .bind {
                        // Do not set broadcast to true so exception is thrown.
                    }
                    .use { socket ->
                        // Send a broadcast which will fail because broadcasts are not enabled on this socket.
                        socket.send(
                            Datagram(
                                packet = buildPacket { writeText("0123456789") },
                                address = NetworkAddress("255.255.255.255", 56700)
                            )
                        )
                    }
            }
        }
    }
}

When running this test the logs show the following:

> Task :ktor-network:jvmTest
Exception in thread "DefaultDispatcher-worker-1 @coroutine#3" java.net.SocketException: Permission denied
	at java.base/sun.nio.ch.DatagramChannelImpl.send0(Native Method)
	at java.base/sun.nio.ch.DatagramChannelImpl.sendFromNativeBuffer(DatagramChannelImpl.java:584)
	at java.base/sun.nio.ch.DatagramChannelImpl.send(DatagramChannelImpl.java:546)
	at java.base/sun.nio.ch.DatagramChannelImpl.send(DatagramChannelImpl.java:529)
	at io.ktor.network.sockets.DatagramSocketImpl.sendImpl(DatagramSocketImpl.kt:98)
	at io.ktor.network.sockets.DatagramSocketImpl$sender$1.invokeSuspend(DatagramSocketImpl.kt:35)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected an exception to be thrown, but was completed successfully.
java.lang.AssertionError: Expected an exception to be thrown, but was completed successfully.
	at org.junit.Assert.fail(Assert.java:88)
	at kotlin.test.junit.JUnitAsserter.fail(JUnitSupport.kt:56)
	at kotlin.test.AssertionsKt__AssertionsKt.checkResultIsFailure(Assertions.kt:153)
	at kotlin.test.AssertionsKt.checkResultIsFailure(Unknown Source)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1$1.invokeSuspend(SocketExceptionIssue.kt:33)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1$1.invoke(SocketExceptionIssue.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:110)
	at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:32)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1.invokeSuspend(SocketExceptionIssue.kt:31)
	(Coroutine boundary)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1.invokeSuspend(SocketExceptionIssue.kt:31)
	(Coroutine creation stacktrace)
	at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:122)
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at io.ktor.network.sockets.tests.SocketExceptionIssue.testBroadcastFails(SocketExceptionIssue.kt:30)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.AssertionError: Expected an exception to be thrown, but was completed successfully.
	at org.junit.Assert.fail(Assert.java:88)
	at kotlin.test.junit.JUnitAsserter.fail(JUnitSupport.kt:56)
	at kotlin.test.AssertionsKt__AssertionsKt.checkResultIsFailure(Assertions.kt:153)
	at kotlin.test.AssertionsKt.checkResultIsFailure(Unknown Source)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1$1.invokeSuspend(SocketExceptionIssue.kt:33)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1$1.invoke(SocketExceptionIssue.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:110)
	at kotlinx.coroutines.TimeoutKt.withTimeout(Timeout.kt:32)
	at io.ktor.network.sockets.tests.SocketExceptionIssue$testBroadcastFails$1.invokeSuspend(SocketExceptionIssue.kt:31)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:81)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:56)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt)
	... 50 more

io.ktor.network.sockets.tests.SocketExceptionIssue[jvm] > testBroadcastFails[jvm] FAILED
    java.lang.AssertionError at SocketExceptionIssue.kt:30
        Caused by: java.lang.AssertionError at SocketExceptionIssue.kt:30
1 test completed, 1 failed

You can see the SocketException in the log and that the test (with assertFails) is unable to catch it.

The following issue looks related: https://youtrack.jetbrains.com/issue/KTOR-440

Here is a fork of Ktor with the test added so you can reproduce the issue: https://github.com/Thomas-Vos/ktor/tree/socket_exception_issue (see last commit for added test). The fork is based on the 1.5.0 branch.

Upgrade kotlinx.serialization to 1.0.0-RC2

ConcurrentList.increaseCapacity() throws ArrayIndexOutOfBoundsException

https://youtrack.jetbrains.com/issue/KTOR-1034

I have a test that uses MockEngine which makes more than 32 calls, and then throws a java.lang.ArrayIndexOutOfBoundsException (stacktrace at the bottom).

I tracked the problem down to io.ktor.util.collections.ConcurrentList.increaseCapacity() (ConcurrentList.kt:180):

    private fun increaseCapacity(targetCapacity: Int = data.size * 2) {
        val newData = SharedList<T>(targetCapacity)
        for (index in 0 until targetCapacity) {
            newData[index] = data[index]
        }

        data = newData
    }

It is clear from this code that the for loop uses targetCapacity which is going to try to access data outside of the original data size. Here is what the code should be:

    private fun increaseCapacity(targetCapacity: Int = data.size * 2) {
        val newData = SharedList<T>(targetCapacity)
        for (index in 0 until data.size) {
            newData[index] = data[index]
        }

        data = newData
    }

Here is the stacktrace for reference.

Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 32 out of bounds for length 32
	at java.base/java.lang.invoke.VarHandle$1.apply(VarHandle.java:2011)
	at java.base/java.lang.invoke.VarHandle$1.apply(VarHandle.java:2008)
	at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:159)
	at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:156)
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:62)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
	at java.base/java.lang.invoke.VarHandleObjects$Array.getVolatile(VarHandleObjects.java:438)
	at java.base/java.lang.invoke.VarHandleGuards.guard_LI_L(VarHandleGuards.java:646)
	at java.base/java.util.concurrent.atomic.AtomicReferenceArray.get(AtomicReferenceArray.java:100)
	at io.ktor.util.collections.internal.SharedList.get(SharedList.kt:35)
	at io.ktor.util.collections.ConcurrentList.increaseCapacity(ConcurrentList.kt:180)
	at io.ktor.util.collections.ConcurrentList.increaseCapacity$default(ConcurrentList.kt:177)
	at io.ktor.util.collections.ConcurrentList.add(ConcurrentList.kt:65)
	at io.ktor.client.engine.mock.MockEngine.execute(MockEngine.kt:63)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:333)
	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:188)
	at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:145)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt:91)
	at kotlinx.coroutines.BuildersKt.async(Unknown Source)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.async$default(Builders.common.kt:84)
	at kotlinx.coroutines.BuildersKt.async$default(Unknown Source)
	at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:81)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:66)
	at io.ktor.client.engine.HttpClientEngine$install$1.invoke(HttpClientEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:188)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:31)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:124)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:88)
	at io.ktor.client.features.HttpSend$Feature$install$1.invoke(HttpSend.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(PipelineContext.kt:178)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:86)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invoke(HttpCallValidator.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:37)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invoke(HttpRequestLifecycle.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:188)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:31)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:185)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at com.gatsby.trade.client.OrbisOrderClient$placeOrder$$inlined$post$1.invokeSuspend(OrbisClient.kt:242)

Dependency for ktor-network 1.4.1 and 1.4.0 doesn't work on Kotlin/Multiplatform

https://youtrack.jetbrains.com/issue/KTOR-1270

Repositories applied: jcenter, mavenCentral
build.gradle.kts:

plugins {
    kotlin("multiplatform") version "1.4.10"
    kotlin("plugin.serialization") version "1.4.10"
}

val ktorVersion = "1.4.1"
val coroutinesVersion = "1.4.1"
val serializationVersion = "1.0.1"

allprojects {

    plugins.apply("org.jetbrains.kotlin.multiplatform")
    plugins.apply("org.jetbrains.kotlin.plugin.serialization")

    repositories {
        mavenCentral()
        jcenter()
    }

    kotlin {

        jvm {
            compilations.all {
                kotlinOptions.jvmTarget = "1.8"
            }
            testRuns["test"].executionTask.configure {
                useJUnitPlatform()
            }
        }
        val hostOs = System.getProperty("os.name")
        val isMingwX64 = hostOs.startsWith("Windows")

        val nativeTarget = when {
            hostOs == "Mac OS X" -> macosX64("native")
            hostOs == "Linux" -> linuxX64("native")
            isMingwX64 -> mingwX64("native")
            else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
        }

        sourceSets {
            val commonMain by getting {
                dependencies {
                    implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:$serializationVersion")
                    implementation("io.ktor:ktor-network:$ktorVersion")
                    implementation("io.ktor:ktor-io:$ktorVersion")
                    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion")
                }
            }
            val commonTest by getting {
                dependencies {
                    implementation(kotlin("test-common"))
                    implementation(kotlin("test-annotations-common"))
                }
            }
            val jvmMain by getting
            val jvmTest by getting {
                dependencies {
                    implementation(kotlin("test-junit5"))
                    implementation("org.junit.jupiter:junit-jupiter-api:5.6.0")
                    runtimeOnly("org.junit.jupiter:junit-jupiter-engine:5.6.0")
                }
            }
            val nativeMain by getting
            val nativeTest by getting
        }
    }

}

error screenshot:

CIO client engine randomly runs into infinite loop under high load

https://youtrack.jetbrains.com/issue/KTOR-1126

Stuck in while (!closed) in io.ktor.network.selector.ActorSelectorManager.process().
Probably related to KTOR-717.
Likely a duplicate of KTOR-531.

I also randomly get this:

Exception in thread "main" java.net.BindException: Can't assign requested address
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Net.java:503)
	at java.base/sun.nio.ch.Net.connect(Net.java:492)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:751)
	at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:31)
	at io.ktor.network.sockets.ConnectUtilsJvmKt.connect(ConnectUtilsJvm.kt:19)
	at io.ktor.network.sockets.TcpSocketBuilder.connect(TcpSocketBuilder.kt:38)
	at io.ktor.client.engine.cio.ConnectionFactory.connect(ConnectionFactory.kt:24)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invokeSuspend(Endpoint.kt:159)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invoke(Endpoint.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:120)
	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:83)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:167)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:95)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more
	Suppressed: java.net.BindException: Can't assign requested address
		... 21 more

Repro:

import io.ktor.application.*
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import kotlin.random.*
import kotlin.time.*
import kotlinx.coroutines.*

@OptIn(ExperimentalTime::class)
suspend fun main() = withContext(Dispatchers.Default) {
    startServer()
    delay(1_000)

    val batchSize = 100
    val threadCount = 1

    HttpClient(CIO) {
        engine {
            threadsCount = threadCount
        }
    }.use { client ->
        repeat(Int.MAX_VALUE) { i ->
            val time = measureTime {
                coroutineScope {
                    repeat(batchSize) {
                        launch {
                            client.get<HttpResponse>("http://localhost:8080").readText()
                        }
                    }
                }
            }

            println("${i + 1}: $time")
        }
    }
}

private fun startServer() {
    embeddedServer(Netty, port = 8080) {
        val random = Random(0)
        val data = buildString(100_000) { repeat(100_000) { append(random.nextInt(0, 10)) } }

        routing {
            get("/") {
                call.respondText(data, ContentType.Text.Plain)
            }
        }
    }.start()
}

Docs

Client HttpCache feature is not documented

Wrong redirect from old documentation page

404 page on https://ktor.io/clients/http-client.html#websockets

clients-index documentation page incorrect link

Apache client engine is documented to support HTTP/2 but doesn't

Public documentation for Ktor 1.4.0 says, that it's in the development

New documentation lacks artifacts information for Gradle and Maven

Docs: Invalid id attribute syntax in html usage example

Generator

Generate endpoint: 500 server errors without meaningful messages about them

Generator API: bad error message if wrong build system or engine provided to /project/generate

On Safari some elements have incorrect size

Infrastructure

Publish changelog on Github releases

Enable binary-compatibility-validator in all modules

ktor npm packages haven't been published since version 1.3.2

Gradle could not find artifact ktor-client-okhttp-1.4.0-samplessources.jar when okhttp specified in common sourceset

IntelliJ IDEA Plugin

Ktor Project Creation Wizard: Generated Code For mock engine not compiling

Plugin: "Could not find io.ktor:ktor-client-http-timeout" when building project created with http-timeout feature

No changelog for 1.4.0 and 1.4.1 on the plugin page in IDE Marketplace

Samples

Routing: get matcher has higher priority than param matcher on the same level

[SECURITY] 0.9.* Releases were built/executed/released in the context of insecure/untrusted code

Please remove or update these samples, because they do not compile with new versions of Kotlin

Form authorization not working in example (feature/auth)

Need help compiling fullstack-mpp.

Missing dependency io.ktor:ktor-client-ios_debug_ios_arm64:1.1.1

Server

Configure feature multiple times

Support application/xml as part for the ContentNegotiation feature

Ktor stopped working with latest Tomcat 9.0.39

Add UUID to DefaultConversionService

Serializing collections of different element types

https://youtrack.jetbrains.com/issue/KTOR-1163

Original question from https://stackoverflow.com/questions/64319112/why-serializing-collections-of-different-element-types-is-not-supported-in-ktor

To reproduce use the following code example:

@Serializable
sealed class Message {
    abstract val content: String
}

@Serializable
data class BroadcastMessage(override val content: String) : Message()

@Serializable
data class DirectMessage(override val content: String, val recipient: String) : Message()

val data: List<Message> = listOf(
    DirectMessage("Hey, Joe!", "Joe"),
    BroadcastMessage("Hey, all!")
)

fun main() {
    embeddedServer(Netty, port = 8080, host = "127.0.0.1") {
        install(ContentNegotiation) {
            json()
        }
        routing {
            get("/") {
                call.respond(data)
            }
        }
    }.start(wait = true)
}

Result:

java.lang.IllegalStateException: Serializing collections of different element types is not yet supported. Selected serializers: [DirectMessage, BroadcastMessage]
	at io.ktor.serialization.SerializerLookupKt.elementSerializer(SerializerLookup.kt:71)
	at io.ktor.serialization.SerializerLookupKt.serializerForSending(SerializerLookup.kt:41)
	at io.ktor.serialization.SerializationConverter.convertForSend(SerializationConverter.kt:114)
	at io.ktor.features.ContentNegotiation$Feature$install$2.invokeSuspend(ContentNegotiation.kt:149)
	at io.ktor.features.ContentNegotiation$Feature$install$2.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at ServerKt$main$1$2$1.invokeSuspend(server.kt:39)
	at ServerKt$main$1$2$1.invoke(server.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.routing.Routing.executeResult(Routing.kt:148)
	at io.ktor.routing.Routing.interceptor(Routing.kt:35)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:100)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:121)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:54)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

Nested Authentication: Only first authentication is executed

Request headers exceeding expected threshold are not handled correctly

Sessions + SSL (Netty)

https://youtrack.jetbrains.com/issue/KTOR-1292

edit: https://github.com/dm6801/ktor_sessions_ssl

when trying to use Session feature with cookies

    install(Sessions) {
        cookie<MySession>("MY_SESSION")
    }

accessing https:/localhost:443/ throws:

java.lang.UnsupportedOperationException: null
	(Coroutine boundary)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:55)
	(Coroutine creation stacktrace)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:188)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:38)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:821)
Caused by: java.lang.UnsupportedOperationException: null
	at io.ktor.server.netty.http2.NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64)
	at io.ktor.sessions.SessionTransportCookie.receive(SessionTransportCookie.kt:28)
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:204)
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:59)
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)

relevant line NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64):

override val cookies: RequestCookies
    get() = throw UnsupportedOperationException()

http://localhost:8080/ - works

Allow any header in CORS configuration

Netty: NumberFormatException when make a request via HTTP/2 without explicit port

https://youtrack.jetbrains.com/issue/KTOR-1091

Hello.

To reproduce it run the server on 443 port and access it without explicit port specifying.

Stack Trace:

java.lang.NumberFormatException: For input string: "localhost"
     	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 	at java.lang.Integer.parseInt(Integer.java:580)
 	at java.lang.Integer.parseInt(Integer.java:615)
 	at io.ktor.server.netty.http2.Http2LocalConnectionPoint.getPort(Http2LocalConnectionPoint.kt:31)
 	at io.ktor.features.HSTS.intercept(HSTS.kt:87)
 	at io.ktor.features.HSTS$Feature$install$1.invokeSuspend(HSTS.kt:101)
 	at io.ktor.features.HSTS$Feature$install$1.invoke(HSTS.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:101)
 	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
 	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:189)
 	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:100)
 	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:140)
 	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
 	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
 	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
 	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:121)
 	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
 	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
 	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
 	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
 	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
 	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:54)
 	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
 	at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:182)
 	at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source)
 	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:145)
 	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
 	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
 	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37)
 	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
 	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
 	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 	at java.lang.Thread.run(Thread.java:748)

The problem is at Http2LocalConnectionPoint:31

I think is would be better to do something like this:

nettyHeaders.authority()?.toString()?.substringAfter(":", if (scheme == "https") "443" else "80")?.toInt()

because often local port information is not enough

Thanks in advance :)

Exception after WebSocketSession.close() invocation.

Multiple matching paths do not choose higher quality path (component with more static values) when the lower quality one is authenticated

Wrong session id get stuck at clients

Confusing log message about failed session lookup

MultiPartData.readAllParts() throws java.io.IOException when multipart list is empty

https://youtrack.jetbrains.com/issue/KTOR-767

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/482

Problem description: When receiving an empty multipart-form request on a POST route, the request handler throws an IOException, preventing graceful handling or the display of an error message. Instead, a 500 error message is returned.

Minimum working example:

import io.ktor.application.*
import io.ktor.content.readAllParts
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.post
import io.ktor.routing.routing

fun main(args: Array<String>): Unit = io.ktor.server.netty.DevelopmentEngine.main(args)

fun Application.module() {
    routing {
        post("/add") {
            if(call.request.isMultipart()) {
                val multipart = call.receiveMultipart()
                val formItems = multipart.readAllParts() // <-- this breaks when sent an empty request
                call.respondText("recvd: $multipart")
            }
            else {
                call.respond("no multipart, fix request!")
            }
        }
    }
}

Send a request with an empty multipart body to see the issue:

curl -X POST \
  http://localhost:8080/add \
  -H 'Cache-Control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'

Error Message:

22:57:11.866 [nettyCallPool-4-4] ERROR Application - Unhandled: POST - /add
java.io.IOException: Broken delimiter occurred
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.doResume(Delimited.kt:57)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt:1746)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiterSuspend(Delimited.kt:55)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiter(Delimited.kt:50)
	at io.ktor.http.cio.MultipartKt.boundary(Multipart.kt:89)
	at io.ktor.http.cio.MultipartKt$parseMultipart$1.doResume(Multipart.kt:158)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
	at kotlinx.coroutines.experimental.DispatchedKt.resumeCancellable(Dispatched.kt:209)
	at kotlinx.coroutines.experimental.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:35)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce(Produce.kt:95)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce$default(Produce.kt:88)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:145)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:138)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:33)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:31)
	at io.ktor.server.engine.DefaultTransformKt.multiPartData(DefaultTransform.kt:70)
	at io.ktor.server.engine.DefaultTransformKt.access$multiPartData(DefaultTransform.kt:1)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.doResume(DefaultTransform.kt:33)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:64)
	at io.sebi.ApplicationKt$module$1$1.doResume(application.kt:31)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.routing.Routing.executeResult(Routing.kt:100)
	at io.ktor.routing.Routing.interceptor(Routing.kt:25)
	at io.ktor.routing.Routing$Feature$install$1.doResume(Routing.kt:66)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt:51)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.doResume(DefaultEnginePipeline.kt:66)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.doResume(NettyApplicationCallHandler.kt:31)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt:10)
	at kotlinx.coroutines.experimental.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:44)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:113)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch(Builders.common.kt:72)
	at kotlinx.coroutines.experimental.BuildersKt.launch(Unknown Source)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:64)
	at kotlinx.coroutines.experimental.BuildersKt.launch$default(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:844)

Netty: Not started servers leak resources

https://youtrack.jetbrains.com/issue/KTOR-939

Netty.create() does some heavy initialization immediately, for example creating EventLoopGroups. These in turn create pipes.
Even though the engine is never started it allocates resources and would only release them when calling stop().

This was quite confusing in unit testing where I create engines repeatedly. They may or may not get started and only get stopped if they were actually started.
I'd expect resources to be only allocated when I call start(), esp. since only stop() will release them.

repeat(1000) {
    embeddedServer(Netty, applicationEngineEnvironment { })
}

Exception in thread "main" java.lang.IllegalStateException: failed to create a child event loop
	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:88)
	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
	at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:52)
	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:96)
	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:91)
	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:72)
	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
	at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:190)
	at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:90)
	at io.ktor.server.netty.Netty.create(Embedded.kt:14)
	at io.ktor.server.netty.Netty.create(Embedded.kt:12)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:79)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:77)
	at server.MainKt.main(Main.kt:15)
	at server.MainKt$$$main.invoke(Unknown Source)
	at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$1.invokeSuspend(IntrinsicsJvm.kt:205)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:115)
	at kotlin.coroutines.jvm.internal.RunSuspendKt.runSuspend(RunSuspend.kt:19)
	at server.MainKt.main(Main.kt)
Caused by: io.netty.channel.ChannelException: failed to open a new selector
	at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:175)
	at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:142)
	at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:146)
	at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:37)
	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
	... 19 more
Caused by: java.io.IOException: Too many open files
	at java.base/sun.nio.ch.IOUtil.makePipe(Native Method)
	at java.base/sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:86)
	at java.base/sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:35)
	at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:173)
	... 23 more

SerializationException when type of serializable class property is generic

https://youtrack.jetbrains.com/issue/KTOR-1200

To reproduce, run the following code and make a request to the /bad endpoint:

@Serializable
class ApiResponse<T>(
    val result: T? = null,
)

fun main() {
    embeddedServer(Netty, port = 8090, host = "127.0.0.1") {
        install(ContentNegotiation) {
            json()
        }
        routing {
            get("/bad") {
                call.respond(ApiResponse(result = "asd"))
            }
        }
    }.start(wait = true)
}

The unexpected exception is logged:

kotlinx.serialization.SerializationException: Serializer for class 'ApiResponse' is not found.
Mark the class as @Serializable or provide the serializer explicitly.
	at kotlinx.serialization.internal.Platform_commonKt.serializerNotRegistered(Platform.common.kt:104)
	at kotlinx.serialization.SerializersKt__SerializersKt.serializer(Serializers.kt:127)
	at kotlinx.serialization.SerializersKt.serializer(Unknown Source)
	at io.ktor.serialization.SerializerLookupKt.serializerForSending(SerializerLookup.kt:59)
	at io.ktor.serialization.SerializationConverter.convertForSend(SerializationConverter.kt:114)
	at io.ktor.features.ContentNegotiation$Feature$install$2.invokeSuspend(ContentNegotiation.kt:149)
	at io.ktor.features.ContentNegotiation$Feature$install$2.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at MainKt$main$1$2$1.invokeSuspend(main.kt:30)
	at MainKt$main$1$2$1.invoke(main.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.routing.Routing.executeResult(Routing.kt:148)
	at io.ktor.routing.Routing.interceptor(Routing.kt:35)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:100)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:113)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:121)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:54)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

Also, I am able to reproduce it in 1.4.2-SNAPSHOT (the latest master branch).

Add excludeSuffix to HttpsRedirect feature

Test Infrastructure

Inconsistent TestApplicationRequest and Client HttpRequestBuilder API's

Replace JUnit with kotlin.test

Other

UDP sockets on native

Getting UnsupportedOperationException when calling call.receive

https://youtrack.jetbrains.com/issue/KTOR-582

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1249

Ktor pre-release version 1.2.3-rc

When trying to parse the request using:
suspend inline fun <reified T : Any> ApplicationCall.receive(): T = receive(T::class)

In this way:
val obj = call.receive<MyCustomObject>()

It produces de following exception:

java.lang.UnsupportedOperationException: This function has a reified type parameter and thus can only be inlined at compilation time, not called directly.
at kotlin.jvm.internal.Intrinsics.throwUndefinedForReified(Intrinsics.java:193)
at kotlin.jvm.internal.Intrinsics.throwUndefinedForReified(Intrinsics.java:187)
at kotlin.jvm.internal.Intrinsics.reifiedOperationMarker(Intrinsics.java:197)
at "my package"$main$2$3.invokeSuspend(KtorServices.kt:205)
at "my package"$main$2$3.invoke(KtorServices.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.routing.Routing.executeResult(Routing.kt:147)
at io.ktor.routing.Routing.interceptor(Routing.kt:34)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:106)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.server.servlet.KtorServlet$blockingService$1.invokeSuspend(KtorServlet.kt:148)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedKt.resumeCancellable(Dispatched.kt:447)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:25)
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:53)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at io.ktor.server.servlet.KtorServlet.blockingService(KtorServlet.kt:113)
at io.ktor.server.servlet.KtorServlet.service(KtorServlet.kt:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at com.google.apphosting.runtime.jetty9.ParseBlobUploadHandler.handle(ParseBlobUploadHandler.java:119)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1182)
at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doHandle(AppEngineWebAppContext.java:187)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:293)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:213)
at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81)
at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:722)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:685)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:655)
at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:847)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:270)
at java.lang.Thread.run(Thread.java:748)

HttpClient can only be used on the main thread for native targets

https://youtrack.jetbrains.com/issue/KTOR-491

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1559

Ktor Version and Engine Used (client or server and name)
Ktor client 1.2.6, default platform engine (iOS)

Describe the bug
HttpClient can only be used from the main thread. It does not matter on which thread the object is instantiated.

The issue is caused by global declarations that are not annotated properly for Kotlin/Native.

Uncaught Kotlin exception: io.ktor.http.URLParserException: Fail to parse url: <redacted>
Caused by: kotlin.native.IncorrectDereferenceException: Trying to access top level value not marked as @ThreadLocal or @SharedImmutable from non-main thread
at 0 test.kexe 0x0000000101edb8e7 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37)
at 1 test.kexe 0x0000000101ed4ee5 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
at 2 test.kexe 0x0000000101ed4aa5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
at 3 test.kexe 0x0000000101f07b55 kfun:kotlin.native.IncorrectDereferenceException.<init>(kotlin.String)kotlin.native.IncorrectDereferenceException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/Runtime.kt:30:36)
at 4 test.kexe 0x0000000101f25e99 ThrowIncorrectDereferenceException + 137 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/internal/RuntimeUtils.kt:87:11)
at 5 test.kexe 0x0000000102200ed9 CheckIsMainThread + 25
at 6 test.kexe 0x00000001020e16a3 kfun:io.ktor.http.<get-URL_ALPHABET_CHARS>#internal + 35 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:11:9)
at 7 test.kexe 0x00000001020e2ecd kfun:io.ktor.http.encodeURLPath@kotlin.String.()kotlin.String + 717 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:59:0)
...

I am using the CoroutineWorker library from Autodesk.

To Reproduce
Steps to reproduce the behavior:

Make a client request from a non-main thread for a native target.

Changelog 1.4 version

1.4.3

Client

Missing Content-Type header in a request

ClassCastException with HTTP client when using via an inline function with reified type parameter

Can I logging single line?

iOS testing MockEngine issue

ClassCastException: kotlin.coroutines.intrinsics.CoroutineSingletons cannot be cast to ...

'%3D' inside query of redirect target location will be replaced to '='

NPE in ApacheRequestProducer when "http://" is requested

Digest authentition: using the realm of the 401 response to an initial incomplete request for an immediate second complete request including realm.

ResponseException is no longer serializable starting from 1.4.0 (breaking change)

ClosedReceiveChannelException when making request with CIO engine using a proxy to https

Client: NPE in FormDataContentKt -> Input.copyTo

CIO: client engine exceptions are both logged and thrown

HttpRedirect feature alters Location header value

CIO Engine's HttpClient may fail when trying to send large size binary data.

Timeout﻿ feature: android engine throws Java's SocketTimeoutException instead of ConnectTimeoutException

Client sometimes doesn't do response validation

Make `body` nullable for request builder

Breaking change: formUrlEncode no longer encodes = in the parameters' values

ktor-client-js - Cannot read property Json of undefined

iOS client fails with CoroutinesInternalError when Logging is used

ktor-client-apache: thread stuck in ByteBufferChannel.readRemainingSuspend

Core

Significant performance penalty with socket write() and file read() on JVM

ByteBufferChannel.readRemaining suspend when should not

Fix socket self connect problem

Incorrect encoding function used for URL path by URLBuilder

Input.readTextExactBytes(n) on empty input different behavior per platform

ktor-io: JVM shared function decrease performance starting from 1.4.0

Experimental API and compatibility guarantees

Docs

java.net.MalformedURLException: unknown protocol: ws KTOR android

OAuth authentication documentation is confusing because of incomplete code example

Full example for configuring response validation is missing

Duplicate entry "Features" in Server docs

Missing dependency information in the Timeout topic

Web feedback from "https://ktor.io/docs/nopage.html"

Guides topic is empty

No content for the Metrics feature topic

Apache client engine is documented to support HTTP/2 but doesn't

Server features list is empty

Missing dependency information the Client Auth topic

Add information about required artifacts to the WebSockets topic

Outdated dependency information in the Client Logging topic

Center the docs

Generator

Ktor generator fails to generate when Auth-Oauth is selected

DoubleReceive and Kotlinx.serialization features are missing in generator

Infrastructure

settings.gradle: if (native_targets_enabled) does nothing

Add build parameter to build ktor with JVM IR compiler

CI does not trigger build on PR to samples repo

Invalid publication 'kotlinMultiplatform' for task :publishKotlinMultiplatformPublicationToMavenLocal on master

IntelliJ IDEA Plugin

Ktor plugin - A new project with Digest authentication uses deprecated API

Server

Memory leak on Https request

Unexpected exception when using Session feature: "Using blocking primitives on this dispatcher is not allowed"

Server losing channel exceptions at receive

Jackson: Crash when sending large responses in 1.4.2

GMTDate: Operator overload (plus/minus) for kotlin.time.Duration

Request headers exceeding expected threshold are not handled correctly

Upgrade Netty to 4.1.54.Final

"Wrong HEX escape": gracefully handle invalid URLs

Netty HTTP/2 HEAD response hangs

Sessions + SSL (Netty)

Support for precompresed files

Digest authentication: cannot successfully pass authentication using curl or web browser

HTML Page response size above 4099 bytes causes fatal sever exceptions

Jackson: Crash when sending large responses in 1.4.3

Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.

Parsing Authorization header throws IllegalArgumentException instead of returning null

Other

Implement Support for Compression Extensions for WebSocket (RFC 7692)

Upgrade gradle to 6.7.1 and enable fs watcher

testWriteXRaceCondition sometimes hangs on CI

Remove copyTo usage from ServerPipeline

Fix failing tests in master after 1.5.0 merge

Timeout feature: android engine throws Java's SocketTimeoutException instead of ConnectTimeoutException