Changelog 1.3 version

1.3.2

released 12th March 2020

Client

OkHttp WebSocket hangs when computer goes to sleep on Linux

Can't set a base url that includes path data

Ktor-client-logging : Add an option to filter logged calls

Can't set FormUrlEncoded header for ktor http client request

Add a Validation phase to the HttpResponsePipeline

Disabling logs from Release builds

Darwin: Allow setting custom NSURLSession

Auto set header from cookie

Customize HttpCache behavior, serve stale if host is down, ignore max-age

java.net.SocketTimeoutException with no lines referencing my code

https://youtrack.jetbrains.com/issue/KTOR-577

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1263

Ktor Version and Engine Used (client or server and name)

io.ktor:ktor-client-core:1.2.3-rc
io.ktor:ktor-client-core-jvm:1.2.3-rc
io.ktor:ktor-client-okhttp:1.2.3-rc

Describe the bug
See the exception below. This exception is reported by Firebase. I can not reproduce it myself although quite some users appear to get this bug. I have no idea where this happens in my code and that is why I am unable to fix it myself. It looks like some sort of timeout error.

Fatal Exception: java.net.SocketTimeoutException: timeout
       at okio.SocketAsyncTimeout.newTimeoutException + 159(SocketAsyncTimeout.java:159)
       at okio.AsyncTimeout.exit$jvm + 203(AsyncTimeout.java:203)
       at okio.AsyncTimeout$source$1.read + 163(AsyncTimeout.java:163)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read + 352(Http1ExchangeCodec.java:352)
       at okhttp3.internal.http1.Http1ExchangeCodec$UnknownLengthSource.read + 488(Http1ExchangeCodec.java:488)
       at okhttp3.internal.connection.Exchange$ResponseBodySource.read + 279(Exchange.java:279)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.cache.CacheInterceptor$cacheWritingResponse$cacheWritingSource$1.read + 158(CacheInterceptor.java:158)
       at okio.RealBufferedSource.read + 148(RealBufferedSource.java:148)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke + 97(OkHttpEngineKt.java:97)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke(OkHttpEngineKt.java:1)
       at kotlinx.coroutines.io.ByteBufferChannel.write + 1508(ByteBufferChannel.java:1508)
       at kotlinx.coroutines.io.ByteWriteChannel$DefaultImpls.write$default + 84(ByteWriteChannel.java:84)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invokeSuspend + 96(OkHttpEngineKt.java:96)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith + 33(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run + 241(DispatchedTask.java:241)
       at java.util.concurrent.ThreadPoolExecutor.runWorker + 1167(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run + 641(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run + 764(Thread.java:764)
Caused by java.net.SocketTimeoutException: Read timed out
       at java.net.SocketInputStream.socketRead0(SocketInputStream.java)
       at java.net.SocketInputStream.socketRead + 119(SocketInputStream.java:119)
       at java.net.SocketInputStream.read + 176(SocketInputStream.java:176)
       at java.net.SocketInputStream.read + 144(SocketInputStream.java:144)
       at okio.InputStreamSource.read + 102(InputStreamSource.java:102)
       at okio.AsyncTimeout$source$1.read + 159(AsyncTimeout.java:159)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read + 352(Http1ExchangeCodec.java:352)
       at okhttp3.internal.http1.Http1ExchangeCodec$UnknownLengthSource.read + 488(Http1ExchangeCodec.java:488)
       at okhttp3.internal.connection.Exchange$ResponseBodySource.read + 279(Exchange.java:279)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.cache.CacheInterceptor$cacheWritingResponse$cacheWritingSource$1.read + 158(CacheInterceptor.java:158)
       at okio.RealBufferedSource.read + 148(RealBufferedSource.java:148)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke + 97(OkHttpEngineKt.java:97)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke(OkHttpEngineKt.java:1)
       at kotlinx.coroutines.io.ByteBufferChannel.write + 1508(ByteBufferChannel.java:1508)
       at kotlinx.coroutines.io.ByteWriteChannel$DefaultImpls.write$default + 84(ByteWriteChannel.java:84)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invokeSuspend + 96(OkHttpEngineKt.java:96)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith + 33(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run + 241(DispatchedTask.java:241)
       at java.util.concurrent.ThreadPoolExecutor.runWorker + 1167(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run + 641(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run + 764(Thread.java:764)

To Reproduce
As the stacktrace shows no lines of my code I do not know how to reproduce it. I would really appreciate some help for this exception. I am not entirely sure if this is related to Ktor, but thought this would be the best place to report it. I have Kotlin Coroutines debug mode enabled as well.

Unable to set the Content-Type header in a request

Strings are not decoded when received as application/json

Base Path/URL for the client

[JS client] Cannot change redirect policy by followRedirects=false

ktor-client-json: TypeInfo for request pipeline

Ktor-Client-Js: Upload ktor-client-js library to npm

Nonce.kt leaks a coroutine

URLConnection works fine but ktor client returns 403 forbidden during a get request

Handle pipeline cancel / error

Exposing some internal classes could help add features to Ktor Http Client

JsonSerializer.write should also receive a type

Websocket CIO engine receive channel may close instantly

Wrong Redirection

Add WebSocket support for the iOS

[client] MPP WebSockets client

Native websocket client support

HTTP-client auth with Bearer token

Retry on HttpCode or network error

ktor-client-cio throws ConnectTimeoutException when attempting to connect to 0.0.0.0

https://youtrack.jetbrains.com/issue/KTOR-395

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1840

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-client-websockets:$ktor_version")
implementation("io.ktor:ktor-client-cio:$ktor_version")

Describe the bug
A program using ktor-client-cio can't connect to localhost via 0.0.0.0. Instead, a ConnectTimeoutException is thrown.

To Reproduce

Checkout the 0-0-0-0-cio-connection-issue branch of the repro project
Run the included EchoServer.kt. This will start a local server that answers websocket requests on :9090/chat with a short text fragment.
Run the included Simplified.kt. This makes a webSocket request inside a runBlocking call in order to retrieve the text fragment from the server.
Instead of retrieving the text snippet, an exception is thrown:

Exception in thread "main" io.ktor.network.sockets.ConnectTimeoutException: Connect timeout has been expired [url=ws://0.0.0.0:9090/chat, connect_timeout=unknown ms]
	at io.ktor.client.features.HttpTimeoutKt.ConnectTimeoutException(HttpTimeout.kt:157)
	at io.ktor.client.features.HttpTimeoutKt.ConnectTimeoutException$default(HttpTimeout.kt:156)
	at io.ktor.client.engine.cio.Endpoint.getTimeoutException(Endpoint.kt:213)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:205)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:93)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior
The WebSocket should connect successfully (like it does when using 127.0.0.1 instead of 0.0.0.0).

If you switch the websocket implementation from ktor-client-cio to ktor-client-okhttp, the client behaves as expected (ignoring my previously reported termination issue)

Support async DNS resolve in Client

http client (native macos_x64) v0.9.5-rc13 Memory leaks found

https://youtrack.jetbrains.com/issue/KTOR-743

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/593

Whenever I try and do a simple http request on macOs the program freezes and reports a memory leak:

an example where I publish the latest eap13 branch commit to localMaven which is tagged (v0.9.5-rc13):

buildscript {
    repositories {
        jcenter()
        maven { url 'https://plugins.gradle.org/m2/' }
        maven { url 'https://dl.bintray.com/jetbrains/kotlin-native-dependencies' }
        maven {url 'http://dl.bintray.com/kotlin/kotlin-dev'}
    }
    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:0.9.2-dev-4008"
    }
}

allprojects {

    project.ext {
        //make extras set inside buildscript available in all modules
        kotlin_version = '1.3.0-rc-51'
        kotlin_coroutines_version = '0.26.1-eap13'
    }
}


apply plugin: 'kotlin-platform-native'

repositories {
    mavenLocal()
    jcenter()
}



components.main {
    targets = ["macos_x64"]
    outputKinds = [EXECUTABLE]
    dependencies {
        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core-native:0.26.1-eap13")


        //you need to clone ktor and publish to local maven
        implementation "io.ktor:ktor-client-ios:0.9.4-SNAPSHOT"
    }

}

example program:

import kotlinx.coroutines.runBlocking
import io.ktor.client.engine.ios.*
import io.ktor.client.*
import io.ktor.http.*
import io.ktor.client.request.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import platform.darwin.*




internal val ApplicationDispatcher: CoroutineDispatcher = NsQueueDispatcher(dispatch_get_main_queue())

fun main(args: Array<String>) = runBlocking<Unit> {
    val api = ApplicationApi()
    api.about {
        println(it)
    }
    delay(3000L)

    return@runBlocking
}

internal class NsQueueDispatcher(
        private val dispatchQueue: dispatch_queue_t
) : CoroutineDispatcher() {
    override fun dispatch(context: CoroutineContext, block: Runnable) {
        dispatch_async(dispatchQueue) {
            block.run()
        }
    }
}

class ApplicationApi {
    private val client = HttpClient()

    fun about(callback: (String) -> Unit) {
        launch(ApplicationDispatcher) {
            val result: String = client.get {
                url {
                    protocol = URLProtocol.HTTPS
                    port = 443
                    host = "tools.ietf.org"
                    encodedPath = "rfc/rfc1866.txt"
                }
            }

            callback(result)
        }
    }
}

when run:

/opt/teamcity-agent/work/4d622a065c544371/runtime/src/main/cpp/Memory.cpp:1124: runtime 
assert: Memory leaks found
Abort trap: 6

Objections to changing boundary to internal on MultiPartFormDataContent?

Sporadic OkHttp errors after upgrading to ktor 1.3.1

https://youtrack.jetbrains.com/issue/KTOR-449

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1708

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-client-core-jvm:1.3.1")
implementation("io.ktor:ktor-client-core:1.3.1")
implementation("io.ktor:ktor-client-jackson:1.3.1")
implementation("io.ktor:ktor-client-logging-jvm:1.3.1")
implementation("io.ktor:ktor-client-okhttp:1.3.1")
implementation("io.ktor:ktor-jackson:1.3.1")
implementation("io.ktor:ktor-metrics-micrometer:1.3.1")
implementation("io.ktor:ktor-metrics:1.3.1")
implementation("io.ktor:ktor-server-host-common:1.3.1")
implementation("io.ktor:ktor-server-netty:1.3.1")

Describe the bug

We've started seeing sporadic OkHttp exceptions in our tests when we upgraded from 1.3.0 to 1.3.1. Upon downgrading back to 1.3.0 (with no other changes) the tests were fine again.

The exception is:

java.io.EOFException: \n not found: limit=0 content=…
	at o.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:240)
	at o.i.h.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242)
	at o.i.h.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:213)
	... 20 common frames omitted
Wrapped by: java.io.IOException: unexpected end of stream on http://called-service/...
	at o.i.h.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:236)
	at o.i.c.Exchange.readResponseHeaders(Exchange.java:115)
	at o.i.h.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.c.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at o.i.c.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at o.i.h.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
	at o.RealCall$AsyncCall.execute(RealCall.java:172)
	at o.i.NamedRunnable.run(NamedRunnable.java:32)
	at j.u.c.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at j.u.c.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:834)

It looks like the HTTP client receives an EOF while reading the response headers from calling http://called-service/... I don't think the culprit is the remote web server though, because this server is called from other services and it's only the one with ktor 1.3.1 that throws these errors. And this one is also fine with ktor 1.3.0.

This issue has already been raised before in the OkHttp repository, for example here.

From what I can see, both ktor 1.3.0 and 1.3.1 both depend on com.squareup.okhttp3:okhttp:3.14.2, so maybe this is due to a change in the way ktor uses OkHttp?

To Reproduce

Not easy to reproduce, unfortunately, because the error only occurs in a small percentage of requests (~5%). I'm hoping someone else observes this issue and can come up with a reproducible test.

XML Support in Ktor

Ktor-client, memory leak

Large responses, ex: java exception stack, seems to not get handled

Ktor Client JS: request to /example requests http://localhost/example

[Question] Override/Add Headers to response to ktor client

ktor-locations for (multiplatform) clients.

java.net.BindException: Address already in use (Windows 10)

Kotlin/Native iOS test freezes on get request with error: NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9807)

ktor-client-okhttp does not terminate after closing WebSocket, gets stuck in waitForReferencePendingList()

HTTP Client does not receive body and gets stuck or silently dies

https://youtrack.jetbrains.com/issue/KTOR-410

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1813

Ktor Version and Engine Used (client or server and name)
1.3.2, client: apache (but same behavior on CIO), server: netty, enabled calls logging
Http client:

fun httpClient() =
    HttpClient(Apache) {
        install(JsonFeature) {
            serializer = JacksonSerializer()
        }

        install(ResponseObserver) {
            onResponse {
                httpClientLogger.trace { "Response received" }
            }
        }

        install(Logging) {
            logger = Logger.TRACE
            level = LogLevel.ALL
        }
    }

Describe the bug
I have following code:

        runCatching {
            client.get<ConversationInformation> {
                url(endpoint)
                header("Authorization", "Bearer $token")
            }
        }.onFailure {
            logger.error(it) { "It was not possible to fetch conversation information!" }
        }.onSuccess {
            logger.debug { "Successfully got conversation information." }
            logger.trace { it }
        }.getOrNull()

And when executed, SOMETIMES it never logs either onFailure nor onSuccess block.
The client GET request in some cases either silently fails or gets stuck/suspended for indefinite time, not sure how is that possible, but following logs are produced by the call logging mentioned previously.

21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - REQUEST: https://<internal URL>
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Authorization: Bearer <some bearer>
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Accept: application/json
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Accept-Charset: UTF-8
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - CONTENT HEADERS
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: null
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging -
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY END
21/04/2020 12:27:55 [TRACE] com.wire.ObserverLogger - Response received
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: application/json
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - RESPONSE: 200 OK
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - FROM: https://<internal URL>
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Date: Tue, 21 Apr 2020 12:27:54 GMT
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Content-Type: application/json
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Vary: Accept-Encoding
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Content-Length: 125
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Via: 1.1 google
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Alt-Svc: clear

It seems like the body was never received.

When the request is successful, log looks like this:

21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - REQUEST: https://<internal URL>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Authorization: Bearer <some bearer>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Accept: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Accept-Charset: UTF-8
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - CONTENT HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: null
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging -
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY END
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Response received
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Sending to registry
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Registered
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - RESPONSE: 200 OK
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - FROM: https://<internal URL>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Date: Tue, 21 Apr 2020 12:43:24 GMT
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Content-Type: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Vary: Accept-Encoding
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Content-Length: 125
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Via: 1.1 google
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Alt-Svc: clear
21/04/2020 12:43:24 [DEBUG] c.w.b.p.services.ConversationService - Successfully got conversation information.
21/04/2020 12:43:24 [TRACE] c.w.b.p.services.ConversationService - ConversationInformationd(dto=SomeDataHere)
21/04/2020 12:43:25 [TRACE] com.wire.HttpCallsLogging - {"id":"3f7fad1d-5066-487d-8836-9e40928bb12f","name":"a","members":[{"id":"420e4b22-fac2-47e8-b07f-63958d7fcccd","status":0}]}
21/04/2020 12:43:25 [TRACE] com.wire.HttpCallsLogging - BODY END

To Reproduce
It is happening for like 1/5 of all requests, whole service code available here

Expected behavior
IF the request is not successful, the client should throw exception or eventually time out and then throw exception and not to silently die. Not sure what is happening though

EDIT:
tried with

            val response = client.get<HttpStatement> {
                url(endpoint)
                header("Authorization", "Bearer $token")
            }.execute()
            logger.trace { "Executed" }

as well and if it gets stuck, Executed is never printed. Also, seems similar to #679

EDIT2:
I found out that the flow is following:

http client executes GET
target server respond with 200
http client gets stuck
target server throws SocketTimeoutException after a while

I this case I would expect exception to be thrown.

EDIT3:
this is definitely connected to

        install(ResponseObserver) {
            onResponse {
                httpClientLogger.trace { "Response received" }
            }
        }

as without observer, everything is nice and shiny

EDIT4:
Happens with Jetty and OkHttp engines as well.
Mostly it happens after multiple requests, like first one or two works, but then it stops working. This is happening in our staging Kubernets.

"kotlin.IllegalStateException: Fail to send body. Content has type: class kotlinx.serialization.json.JsonObject, but OutgoingContent expected." when acceptContentTypes and contentType are differents

https://youtrack.jetbrains.com/issue/KTOR-433

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1745

Ktor Version and Engine Used (client or server and name)
1.3.2, client Curl or IOS

Describe the bug
IllegalStateException is thrown when acceptContentTypes and contentType are different.
When both are same, the client call is ok and http response is correct.

To Reproduce
Steps to reproduce the behavior:

@Serializable
data class User(
    val email: String,
    val password: String
)

@UnstableDefault
@KtorExperimentalAPI
class Sample {
    val client: HttpClient by lazy {
        HttpClient(Curl.create()) {
            expectSuccess = false

            install(JsonFeature) {
                acceptContentTypes = listOf(
                    ContentType.parse("application/vnd.any.response+json")
                )
                serializer = KotlinxSerializer()
            }
        }
    }
}

@ImplicitReflectionSerializer
@KtorExperimentalAPI
@UnstableDefault
fun main() = runBlocking {
    val response: HttpResponse = Sample().client.put {
        url("https://httpbin.org/put")
        contentType(
            ContentType.parse("application/vnd.any+json")
        )
        body = Json(JsonConfiguration.Stable).toJson(User("mail@mail.com", "password1234"))
    }

    println(response.readText())
}

Uncaught Kotlin exception: kotlin.IllegalStateException: Fail to send body. Content has type: class kotlinx.serialization.json.JsonObject, but OutgoingContent expected.
        at 0   untitled.kexe                       0x00000001002e6317 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37)
        at 1   untitled.kexe                       0x00000001002df5d5 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
        at 2   untitled.kexe                       0x00000001002df775 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
        at 3   untitled.kexe                       0x00000001002dfc35 kfun:kotlin.IllegalStateException.<init>(kotlin.String?)kotlin.IllegalStateException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:70:44)
        at 4   untitled.kexe                       0x00000001005d0206 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invokeSuspend#internal + 5270 (/Users/teamcity3/buildAgent/work/4d622a065c544371/backend.native/build/stdlib/kotlin/util/Preconditions.kt:98:15)
        at 5   untitled.kexe                       0x00000001005d0728 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invoke#internal + 312 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:72:71)
        at 6   untitled.kexe                       0x0000000100549d22 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1122 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:207:0)
        at 7   untitled.kexe                       0x000000010054937b kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:18:0)
        at 8   untitled.kexe                       0x00000001005ccdec kfun:io.ktor.client.features.HttpRequestLifecycle.Feature.$install$lambda-0COROUTINE$25.invokeSuspend#internal + 1212 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpRequestLifecycle.kt:34:21)
        at 9   untitled.kexe                       0x00000001005cd508 kfun:io.ktor.client.features.HttpRequestLifecycle.Feature.$install$lambda-0COROUTINE$25.invoke#internal + 312 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpRequestLifecycle.kt:28:73)
        at 10  untitled.kexe                       0x0000000100549d22 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1122 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:207:0)
        at 11  untitled.kexe                       0x000000010054937b kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:18:0)
        at 12  untitled.kexe                       0x00000001005497da kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal + 474 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:183:16)
        at 13  untitled.kexe                       0x0000000100543909 kfun:io.ktor.util.pipeline.Pipeline.execute(TContext;TSubject)TSubject + 361 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/Pipeline.kt:27:41)
        at 14  untitled.kexe                       0x00000001005a44ed kfun:io.ktor.client.HttpClient.$executeCOROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 653 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:164:25)
        at 15  untitled.kexe                       0x00000001005a4834 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:163:13)
        at 16  untitled.kexe                       0x00000001005e0c0f kfun:io.ktor.client.statement.HttpStatement.$executeUnsafeCOROUTINE$96.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 943 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:104:27)
        at 17  untitled.kexe                       0x00000001005e0f1b kfun:io.ktor.client.statement.HttpStatement.executeUnsafe$ktor-client-core()io.ktor.client.statement.HttpResponse + 235 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:101:22)
        at 18  untitled.kexe                       0x00000001005dfbbb kfun:io.ktor.client.statement.HttpStatement.$executeCOROUTINE$93.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 1259 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:43:38)
        at 19  untitled.kexe                       0x00000001005e0584 kfun:io.ktor.client.statement.HttpStatement.execute(kotlin.coroutines.SuspendFunction1<io.ktor.client.statement.HttpResponse,T>){0<kotlin.Any?>}Generic + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:42:13)
        at 20  untitled.kexe                       0x00000001005e06f4 kfun:io.ktor.client.statement.HttpStatement.execute()io.ktor.client.statement.HttpResponse + 228 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:58:43)
        at 21  untitled.kexe                       0x000000010028c905 kfun:sample.$main$lambda-0COROUTINE$0.invokeSuspend#internal + 6117 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:71:32)
        at 22  untitled.kexe                       0x00000001003068dc kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 700 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:26:0)
        at 23  untitled.kexe                       0x0000000100467d0a kfun:kotlinx.coroutines.DispatchedTask.run() + 2570 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:42:0)
        at 24  untitled.kexe                       0x000000010044241d kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()kotlin.Long + 813 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:272:20)
        at 25  untitled.kexe                       0x00000001004748f8 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 1864 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:65:44)
        at 26  untitled.kexe                       0x0000000100473aaf kfun:kotlinx.coroutines.runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>){0<kotlin.Any?>}Generic + 1231 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50:22)
        at 27  untitled.kexe                       0x0000000100473fca kfun:kotlinx.coroutines.runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>;kotlin.Int){0<kotlin.Any?>}Generic + 330 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:33:8)
        at 28  untitled.kexe                       0x000000010028aff2 kfun:sample.main() + 162 (/Users/niccheva/code/kotlin/untitled/src/macosMain/kotlin/sample/SampleMacos.kt:47:14)
        at 29  untitled.kexe                       0x000000010028e06a Konan_start + 138 (/Users/niccheva/code/kotlin/untitled/src/macosMain/kotlin/sample/SampleMacos.kt:47:1)

Expected behavior
Expected client call to works when accept and content type are different.

Ktor client raises "ClosedSelectorException" instead of proper error

https://youtrack.jetbrains.com/issue/KTOR-452

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1696

Ktor Version and Engine Used (client or server and name)
Client 1.3.1, JVM 11

Describe the bug
Sometimes Ktor Http client raises ClosedSelectorException for tcp connectivity issues. Looks like (I'm not sure) this error is the result of overloaded server, which opens tcp channels for too long time.

To Reproduce
Unfortunately, there aren't exact steps, which lead to error.

Prepare overloaded slow server on nginx (limit tcp connections also).
Construct Ktor Client.
Execute the following:

val response = client.request<HttpResponse>(url)

if (status.isSuccess()) {
    return ClientResponse.Success(response.readText())
}
return null

Expected behavior
String with response is returned OR error with clear text/type is raised (IOException, TimeoutCancellationException, etc.)

Actual behavior
The following error is raised sometimes:

java.nio.channels.ClosedSelectorException
	at io.ktor.network.selector.ActorSelectorManager.publishInterest(ActorSelectorManager.kt:145)
	at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.kt:47)
	at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:33)
	at io.ktor.network.sockets.TcpSocketBuilder.connect(Builders.kt:107)
	at io.ktor.client.engine.cio.ConnectionFactory.connect(ConnectionFactory.kt:26)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invokeSuspend(Endpoint.kt:159)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invoke(Endpoint.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:78)
	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:57)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:167)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:93)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:561)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:727)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:667)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:655)

So from API user it is unable to proper understand, what was happened. Moreover, this error shows some integrity fails (not sure), however only official wrappers were used (see readText function)

Crash on multiple requests

https://youtrack.jetbrains.com/issue/KTOR-543

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1395

Ktor Version and Engine Used (client or server and name)
Common:
using
apply plugin: 'kotlinx-serialization'

dependencies:
io.ktor:ktor-client-core:1.2.5
io.ktor:ktor-client-json:1.2.5
io.ktor:ktor-client-serialization:1.2.5
org.jetbrains.kotlinx:kotlinx-serialization-runtime-common:0.13.0
Android:
io.ktor:ktor-client-android:1.2.5
io.ktor:ktor-client-json-jvm:1.2.5
io.ktor:ktor-client-serialization-jvm:1.2.5

Describe the bug
Requesting multiple times in a short period (e.g. concurrent calls) from the library causes a crash.

To Reproduce
Steps to reproduce the behavior:

Creating a button to load any url, click it multiple times quickly, eventually it will fail
Using K/N and kodein

bind<HttpClient>() with provider {
        HttpClient {
            install(JsonFeature) {
                serializer = KotlinxSerializer(Json.nonstrict).apply {
                    setMapper(ApiResponse::class, ApiResponse.serializer())
                }

            }
            install(HttpCallValidator)
        }
    }

And to consume it:

override suspend fun getResponse(): ApiResponse  {
        val apiResponse = client.get<ApiResponse> {
            apiUrl("random")
        }
        return ApiResult.FetchedReponse(apiResponse)
    }
}

See error

Process: com.vonderful.testapi.lib, PID: 19866
    io.ktor.client.call.NoTransformationFoundException: No transformation found: class kotlinx.coroutines.io.ByteBufferChannel (Kotlin reflection is not available) -> class com.vonderful.testapi.data.ApiResponse (Kotlin reflection is not available)
    at io.ktor.client.call.HttpClientCall.receive(HttpClientCall.kt:88)
    at com.vonderful.testapi.data.RemoteApiDataSourceImpl.getRandom(RemotePhotoApiDataSource.kt:48)
    at com.vonderful.testapi.data.RemoteApiDataSourceImpl$getRandom$1.invokeSuspend(Unknown Source:11)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)

Expected behavior
Even with a debounce system of 500 the io.ktor.client.call.NoTransformationFoundException: No transformation found: class kotlinx.coroutines.io.ByteBufferChannel (Kotlin reflection is not available) is quite easy to reproduce...

String can not be cast to io.ktor.client.call.HttpClientCall

https://youtrack.jetbrains.com/issue/KTOR-576

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1265

Ktor Version and Engine Used (client or server and name)
1.2.3

Describe the bug
We are trying to create our project on kotlin multiplatform. Now trying to create architecture. Firstly I was getting 415 from server. After I changed content-type I started to get this error.

java.lang.String cannot be cast to io.ktor.client.call.HttpClientCall

It does not crash and no more logs included. I am using okhttp client. Before setting content-type, I could see the logs of my request. But after setting it I do not see those logs only the error above. It looks like it does not even make request because of this error.

To Reproduce
gradle file

kotlin {
    targets {
        fromPreset(presets.jvm, 'android')
    }

    sourceSets {
        commonMain.dependencies {
            api 'org.jetbrains.kotlin:kotlin-stdlib-common'

            implementation "io.ktor:ktor-client:1.2.3"
            implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-common:1.0.1"
            implementation "io.ktor:ktor-client-json:1.2.3"
            implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.11.0"
        }

        androidMain.dependencies {
            api 'org.jetbrains.kotlin:kotlin-stdlib'

            implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.11.0"
            implementation "io.ktor:ktor-client-serialization-jvm:1.2.3"
            implementation 'io.ktor:ktor-client-okhttp:1.2.3'
            implementation "com.squareup.okhttp3:logging-interceptor:4.0.1"
            implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.0.1"
        }
    }
}

engine

actual val clientEngine = HttpClient(OkHttp){
    install(JsonFeature) {
        serializer = KotlinxSerializer()
    }
    engine {
        val loggingInterceptor = HttpLoggingInterceptor()
        loggingInterceptor.level = HttpLoggingInterceptor.Level.BODY
        addInterceptor(loggingInterceptor)
    }

}.engine

request

suspend fun <T> makeRequest(requestMethod: HttpMethod, requestPath: String, requestModel: SerializationStrategy<T>, obj: T): HttpResponse {
        val json = Json(JsonConfiguration.Stable)
        val bodyStr = json.stringify(requestModel, obj)

        return client.request {
            url {
                protocol = URLProtocol.HTTPS
                method = requestMethod
                host = baseUrl
                encodedPath = requestPath
                body = bodyStr
                headers {
                    header("Content-Type", "application/json")
                }
            }
        }
    }

Expected behavior
not giving error and making request

If you need more info let me know.

OkHttp WebSocket terminates with a delay

DoubleReceiveException inside HttpCallValidator.validateResponse

Headers use encoding different from other major HTTP clients (engine-default vs. UTF-8)

Unexpected behavior of the Android client on Android L

Getting crash "Trust anchor for certification path not found"

Android client requires packagingOptions

No proper way to specify subprotocol in WebSocket client without breaking install(Auth)

https://youtrack.jetbrains.com/issue/KTOR-672

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/940

Ktor Version

1.1.2

Ktor Engine Used(client or server and name)

CIO (currently the only available WebSocket client provided by Ktor)

JVM Version, Operating System and Relevant Context

Java 11, Linux

Feedback

Currently, as far as I could find, there is apparently no convenient way in the Ktor WebSocket client API to specify the subprotocol(s) to connect with. I'm talking about the value of the "Sec-WebSocket-Protocol" header. Ktor provides a way to specify this when you set up a WebSocket server, through the protocol argument in Route.webSocket(). On the client side however, there is no such argument. I would expect HttpClient.ws and HttpClient.wss to have a similar "protocol" argument as well, but they don't.

As a workaround you can explicitly pass on an HttpRequestBuilder instance to those functions as the "request" argument, in which you then explicitly set the "Sec-WebSocket-Protocol" header (available as the value HttpHeaders.SecWebSocketProtocol in the Ktor API), but as soon as you do so, this overrides any HTTP authentication that you installed in the HttpClient by invoking install(Auth). So subsequently, you then also have to manually encode the username and password in the case of basic authentication. I expect that you'd have to jump through even more hoops in the case of HTTP Digest authentication.

Ideally, I would like to see an optional protocol (String?) argument added to the HttpClient.ws and HttpClient.wss functions to take the "Sec-WebSocket-Protocol" stuff out of our hands, while still being compatible with the install(Auth) function when configuring the HttpClient.

Below is code that reproduces the issue I'm describing here. If you comment out the "header(HttpHeaders.Authorization, ...)" part, you'll see that the "install(Auth) { basic { ... } }" stuff will be ignored when performing a WebSocket upgrade. Be sure to change the "host" and "path" argument to point to some actually running WebSocket server (and if necessary change "client.wss" to "client.ws") to run this code.

import io.ktor.client.HttpClient
import io.ktor.client.features.auth.Auth
import io.ktor.client.features.auth.providers.basic
import io.ktor.client.features.websocket.WebSockets
import io.ktor.client.features.websocket.wss
import io.ktor.client.request.header
import io.ktor.http.HttpHeaders
import io.ktor.http.HttpMethod
import io.ktor.http.cio.websocket.Frame
import io.ktor.http.cio.websocket.readText
import kotlinx.coroutines.channels.filterNotNull
import kotlinx.coroutines.channels.map
import kotlinx.coroutines.runBlocking
import java.util.Base64

val httpBasicAuthName = "username"
val httpBasicPassword = "password"

private val client = HttpClient().config {

    // NOTE: install(Auth) doesn't actually work when passing on a custom request argument to client.wss().
    install(Auth) {
        basic {
            username = httpBasicAuthName
            password = httpBasicPassword
        }

        install(WebSockets)
    }

}

runBlocking {
    client.wss(
        method = HttpMethod.Get,
        host = "somedomain.com",
        path = "/some-endpoint",
        request = {

            // See https://www.iana.org/assignments/websocket/websocket.xml#subprotocol-name
            header(HttpHeaders.SecWebSocketProtocol, "ocpp2.0,ocpp1.6")

            /* NOTE: Using install(Auth) in HttpClient().config doesn't work here for HTTP Basic Authentication,
             * since we're passing in this explicit request lambda argument, because we need to specify the
             * "Sec-WebSocket-Version" header.
             */
            header(
                HttpHeaders.Authorization,
                "Basic ${Base64.getEncoder().encodeToString(
                    "$httpBasicAuthName:$httpBasicPassword".toByteArray(
                        Charsets.UTF_8
                    )
                )}"
            )

        }) {
        // this: DefaultClientWebSocketSession
        send(Frame.Text("Hello World"))

        for (message in incoming.map { it as? Frame.Text }.filterNotNull()) {
            println(message.readText())
        }
    }
}

Allow modification of response during client "receive" interception

[Client] GET Request result in error with body changed

Multipart form file send writeFully fails with "java.net.ProtocolException: unexpected end of stream"

https://youtrack.jetbrains.com/issue/KTOR-402

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1829

ktorVersion = 1.3.2
implementation "io.ktor:ktor-client-core:$ktorVersion"
implementation "io.ktor:ktor-client-okhttp:$ktorVersion"
implementation "com.squareup.okhttp3:logging-interceptor:4.5.0"

sending file in form, code below works (file fitting the buff)

        val retMessage =  client.request<Message> {
            method = HttpMethod.Patch
            url("https://myhost/rest/v1/message/image-upload/aa")

            header("X-Client-Locale", settings.locale)
            header("Authorization", "Bearer ${settings.jwt}")
            userAgent(settings.userAgent)

            body = MultiPartFormDataContent(
                formData {
                    val filename = reader.open()
                    val buff = ByteArray(102400)
                    val sz = reader.read(buff)
                    if (sz > 0) {
                        append(
                            "picture",
                            filename,
                            ContentType.parse(reader.contentType),
                            size = reader.size
                        ) {
                            writeFully(buff, 0, sz)
                        }
                    }
                    reader.close()
                }
            )
        }

when I move reader.read into BytePacketBuilder

val buff = ByteArray(102400)
                    append(
                        "picture",
                        filename,
                        ContentType.parse(reader.contentType),
                        size = reader.size
                    ) {
                        val sz = reader.read(buff)
                        if (sz > 0) {
                            writeFully(buff, 0, sz)
                        }
                    }

getting:

    ---3e33c4c9bf0fe2451394e9a-21e8f6434bf41b1b-3454ee4d-35548052-6fcbb83a7a--
    --> END PATCH (26800-byte body)
D/OkHttp: <-- HTTP FAILED: java.net.ProtocolException: unexpected end of stream
E/LonjeShared: Rest.request(/rest/v1/message/image-upload/aa): java.net.ProtocolException: unexpected end of stream

same issue happens with InputProvider

                    val filename = reader.open()
                    val inp = reader.input
                    if (inp != null) {
                        append(
                            "picture",
                            InputProvider(reader.size) {
                                return@InputProvider inp
                            }
                        )
                    }

it shows appropriate HTTP PATCH content in OkHttp log

please help me find an issue, I hope I'm doing something wrong. But it can be a serious ktor issue.

PS: on serverside I'm getting:

ERRO multipart: NextPart: stream error: stream ID 7; CANCEL
INFO Mon, 27 Apr 2020 21:00:03 UTC 76.77.185.144   400 PATCH https://myhost/rest/v1/message/image-upload/aa HTTP/2.0   33B in 1.351711ms

what shows HTTP2 stream exception

The DefaultRequest feature overwrites any user specific data

Last-Modified headers are ignored unless set within ConditionalHeaders { version }

Websocket feature does not fully clean itself up on client close

crypto is undefined in IE11

Apache HTTP Client does not send Content-Length header if body is empty content

Client upload/download progress observer/handler/interceptor

Remove dependency on 'text-encoding' npm package

Jetty: requests to resources, that doesn't respond with HTTP/2, lead to unexpected behaviour

https://youtrack.jetbrains.com/issue/KTOR-874

To reproduce run the following code:

fun main() {
    val client = HttpClient(Jetty) {}
    runBlocking {
        val r = client.get<String>("http://www.google.com")
        println(r)
    }
}

As a result, org.eclipse.jetty.io.EofException will be thrown:

Exception in thread "main" org.eclipse.jetty.io.EofException
	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:283)
	at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
	at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
	at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
	at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:259)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
	at org.eclipse.jetty.http2.HTTP2Session.newStream(HTTP2Session.java:543)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$jettyRequest$1.invoke(JettyHttpRequest.kt:40)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$jettyRequest$1.invoke(JettyHttpRequest.kt)
	at io.ktor.client.engine.jetty.UtilsKt.withPromise(utils.kt:14)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.executeRequest(JettyHttpRequest.kt:39)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$1.invokeSuspend(JettyHttpRequest.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.nio.channels.AsynchronousCloseException
	at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:501)
	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:263)
	... 18 more

If I change the URL to http://example.com then the app will hang indefinitely, here is a stacktrace:

"main" #1 prio=5 os_prio=0 cpu=337.00ms elapsed=56.16s tid=0x00007f8e98018800 nid=0x79547 waiting on condition  [0x00007f8e9eb77000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x00000006890f1518> (a kotlinx.coroutines.BlockingCoroutine)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:87)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at MainKt.main(main.kt:86)
	at MainKt.main(main.kt)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.36ms elapsed=56.11s tid=0x00007f8e98311000 nid=0x79550 waiting on condition  [0x00007f8e80382000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.7/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@11.0.7/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.7/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=1.63ms elapsed=56.11s tid=0x00007f8e98315000 nid=0x79551 in Object.wait()  [0x00007f8e80281000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.7/Native Method)
	- waiting on <0x00000006890f2008> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006890f2008> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.7/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.22ms elapsed=56.10s tid=0x00007f8e9832a000 nid=0x79552 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 cpu=497.80ms elapsed=56.10s tid=0x00007f8e9832c000 nid=0x79553 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=306.59ms elapsed=56.10s tid=0x00007f8e9832e000 nid=0x79554 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #7 daemon prio=9 os_prio=0 cpu=6.70ms elapsed=56.09s tid=0x00007f8e98330000 nid=0x79555 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #8 daemon prio=8 os_prio=0 cpu=0.37ms elapsed=56.03s tid=0x00007f8e98381800 nid=0x79556 in Object.wait()  [0x00007f8e47efd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.7/Native Method)
	- waiting on <0x00000006890f27e0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006890f27e0> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@11.0.7/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)
	at jdk.internal.misc.InnocuousThread.run(java.base@11.0.7/InnocuousThread.java:134)

"Monitor Ctrl-Break" #9 daemon prio=5 os_prio=0 cpu=8.39ms elapsed=55.91s tid=0x00007f8e98695800 nid=0x79557 runnable  [0x00007f8e479d4000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(java.base@11.0.7/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.7/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:140)
	at sun.nio.cs.StreamDecoder.readBytes(java.base@11.0.7/StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(java.base@11.0.7/StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(java.base@11.0.7/StreamDecoder.java:178)
	- locked <0x00000006890f4d00> (a java.io.InputStreamReader)
	at java.io.InputStreamReader.read(java.base@11.0.7/InputStreamReader.java:185)
	at java.io.BufferedReader.fill(java.base@11.0.7/BufferedReader.java:161)
	at java.io.BufferedReader.readLine(java.base@11.0.7/BufferedReader.java:326)
	- locked <0x00000006890f4d00> (a java.io.InputStreamReader)
	at java.io.BufferedReader.readLine(java.base@11.0.7/BufferedReader.java:392)
	at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:61)

"Service Thread" #10 daemon prio=9 os_prio=0 cpu=0.04ms elapsed=55.91s tid=0x00007f8e98696800 nid=0x79558 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"ktor-jetty-dispatcher-worker-1" #11 daemon prio=5 os_prio=0 cpu=230.54ms elapsed=55.68s tid=0x00007f8e989de000 nid=0x7955a waiting on condition  [0x00007f8e46ec0000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:357)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.park(CoroutineScheduler.kt:783)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.tryPark(CoroutineScheduler.kt:728)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:711)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

"ktor-jetty-dispatcher-worker-2" #12 daemon prio=5 os_prio=0 cpu=9.27ms elapsed=55.68s tid=0x00007f8e989df000 nid=0x7955b waiting on condition  [0x00007f8e46dbf000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:357)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.park(CoroutineScheduler.kt:783)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.tryPark(CoroutineScheduler.kt:728)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:711)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

"ktor-jetty-client-qtp-13" #13 prio=5 os_prio=0 cpu=6.95ms elapsed=55.47s tid=0x00007f8e340da000 nid=0x7955d runnable  [0x00007f8e46575000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPoll.wait(java.base@11.0.7/Native Method)
	at sun.nio.ch.EPollSelectorImpl.doSelect(java.base@11.0.7/EPollSelectorImpl.java:120)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@11.0.7/SelectorImpl.java:124)
	- locked <0x000000069f414300> (a sun.nio.ch.Util$2)
	- locked <0x000000069f4142a8> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(java.base@11.0.7/SelectorImpl.java:141)
	at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:472)
	at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:409)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
	at org.eclipse.jetty.io.ManagedSelector$$Lambda$65/0x0000000800176c40.run(Unknown Source)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-14" #14 prio=5 os_prio=0 cpu=126.50ms elapsed=55.47s tid=0x00007f8e340dc000 nid=0x7955e waiting on condition  [0x00007f8e46474000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-15" #15 prio=5 os_prio=0 cpu=0.27ms elapsed=55.47s tid=0x00007f8e340dd800 nid=0x7955f waiting on condition  [0x00007f8e46373000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069fc076c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.7/AbstractQueuedSynchronizer.java:2211)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:309)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:379)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-16" #16 prio=5 os_prio=0 cpu=2.68ms elapsed=55.46s tid=0x00007f8e340df800 nid=0x79560 waiting on condition  [0x00007f8e46272000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-17" #17 prio=5 os_prio=0 cpu=0.10ms elapsed=55.46s tid=0x00007f8e340e1000 nid=0x79561 waiting on condition  [0x00007f8e46171000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-18" #18 prio=5 os_prio=0 cpu=0.07ms elapsed=55.46s tid=0x00007f8e340e3000 nid=0x79562 waiting on condition  [0x00007f8e46070000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-19" #19 prio=5 os_prio=0 cpu=0.07ms elapsed=55.46s tid=0x00007f8e340e5000 nid=0x79563 waiting on condition  [0x00007f8e45f6f000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-20" #20 prio=5 os_prio=0 cpu=0.06ms elapsed=55.46s tid=0x00007f8e340e7000 nid=0x79564 waiting on condition  [0x00007f8e45e6e000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"Scheduler-779376001-1" #21 prio=5 os_prio=0 cpu=10.86ms elapsed=55.42s tid=0x00007f8e34117000 nid=0x79565 waiting on condition  [0x00007f8e45b6d000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f415c20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.7/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.7/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.7/ScheduledThreadPoolExecutor.java:1170)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.7/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.7/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.7/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.7/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"Attach Listener" #22 daemon prio=9 os_prio=0 cpu=8.56ms elapsed=55.08s tid=0x00007f8e4805c800 nid=0x79567 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=16.20ms elapsed=56.11s tid=0x00007f8e98309000 nid=0x7954f runnable  

"GC Thread#0" os_prio=0 cpu=22.88ms elapsed=56.16s tid=0x00007f8e98041800 nid=0x7954a runnable  

"GC Thread#1" os_prio=0 cpu=22.20ms elapsed=55.66s tid=0x00007f8e5c001000 nid=0x7955c runnable  

"G1 Main Marker" os_prio=0 cpu=0.51ms elapsed=56.15s tid=0x00007f8e980cc800 nid=0x7954b runnable  

"G1 Conc#0" os_prio=0 cpu=0.07ms elapsed=56.14s tid=0x00007f8e980ce000 nid=0x7954c runnable  

"G1 Refine#0" os_prio=0 cpu=0.50ms elapsed=56.13s tid=0x00007f8e9823b800 nid=0x7954d runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=10.66ms elapsed=56.13s tid=0x00007f8e9823d800 nid=0x7954e runnable  
"VM Periodic Task Thread" os_prio=0 cpu=42.21ms elapsed=55.91s tid=0x00007f8e98698000 nid=0x79559 waiting on condition  

JNI global refs: 14, weak refs: 0

If I make the same requests with curl: curl --http2-prior-knowledge http://example.com then I get an understandable error:

curl: (16) Error in the HTTP2 framing layer

Support Java HTTP Client

MultiPartFormDataContent adds extra lines to output which breaks Apache server

Content-type header is missing in Request when using LogLevel.BODY

Ktor Client HTTP Cache for invalid or blank Expires header

https://youtrack.jetbrains.com/issue/KTOR-364

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1892

Ktor Version and Engine Used (client or server and name)
1.3.2, client, io.ktor.client.features.cache.HttpCache

Describe the bug
If a server returns a blank or an invalid Expires HTTP header, the ktor client crashes if the feature HttpCache is enabled install(HttpCache)

To Reproduce
Steps to reproduce the behavior:

Write the following:

val articleUrl = Url("https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503")
val newsClient = HttpClient(Apache) {
    install(HttpCache)

    install(UserAgent) {
        agent = "Mozilla/5.0"
    }
}
try {
    val call: HttpStatement = newsClient.get(articleSimilarityRequest.articleUrl)
    val response: HttpResponse = call.receive()
    val requestUrl = response.request.url.toString()
    val rawHtml = response.readText()
    println(rawHtml)
} catch (cause: Throwable) {
    println("Unable to download article")
} finally {
    newsClient.close()
}

If you open the URL https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503 in the browser, you can see that the Expires response header is blank
An exception is thrown if you run the code:

2020-05-23 14:41:24.124 [DefaultDispatcher-worker-2 @request#48] ERROR a.s.api.util.service.DownloadService - Unable to download article
java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	(Coroutine boundary)
	at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:123)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:116)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:89)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
	at app.shwoo.api.util.service.DownloadService.downloadArticle$suspendImpl(DownloadService.kt:101)
	at app.shwoo.api.server.web.SimilarityResourceKt$similarity$1$4.invokeSuspend(SimilarityResource.kt:38)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
	at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:292)
	at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
	at io.ktor.server.testing.TestApplicationEngine$handleRequest$pipelineJob$1.invokeSuspend(TestApplicationEngine.kt:290)
Caused by: java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior
It seems that the problem occurs in https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/features/cache/HttpCacheEntry.kt#L84. If the Expires header is blank an exception is thrown. The expected behavior is that the application handles the cache as expired if an invalid date is provided in the HTTP Expires header. Like described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires

Add response text to the message of ResponseException and derived exceptions

Parser Exception in header with character code 1 not allowed

https://youtrack.jetbrains.com/issue/KTOR-860

The following URL

https://data.smartdublin.ie/cgi-bin/rtpi

returns valid results with other HTTP clients, including IntelliJ HTTP Requests. However with Ktor, independently of engine, it throws a parser exception (side note, it would be good to indicate the header in the exception result)

Exception in thread "main" io.ktor.http.cio.ParserException: Character with code 1 is not allowed in header names, 
HTTP/1.1 200 OKServer: nginx/1.17.8Date: Sun, 19 Jul 2020 07:12:42 GMTContent-Type: application/json; charset=utf-8Content-Length: 1596054Connection: keep-aliveX-Powered-By: ASP.NETSet-Cookie: visid_incap_1822411=6ycr4UzsSv+6sdm+QnNmpmnyE18AAAAAQUIPAAAAAACm9JYEnYu4a0fTA7VxKyxu; expires=Sun, 18 Jul 2021 08:40:35 GMT; HttpOnly; path=/; Domain=.nationaltransport.ieSet-Cookie: incap_ses_535_1822411=vjDoPwoqvTdguklauLNsB2nyE18AAAAAoieq1zxaD46bWhzEJyQd3w==; path=/; Domain=.nationaltransport.ieSet-Cookie: ___utmvmzauvysSB=WQYQExBjiZv; path=/; Max-Age=900Set-Cookie: ___utmvazauvysSB=TjhnJRz; path=/; Max-Age=900
	at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:279)
	at io.ktor.http.cio.HttpParserKt.parseHeaderValue(HttpParser.kt:261)
	at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:119)
	at io.ktor.http.cio.HttpParserKt.parseResponse(HttpParser.kt:73)
	at io.ktor.http.cio.HttpParserKt$parseResponse$1.invokeSuspend(HttpParser.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Process finished with exit code 1

To reproduce see attached code.

ClassCastException when POSTing an object without explicitly setting a Content-Type

httpClientCall cannot be canceled in other job

Add tracing to Apache HttpClient engine

404 doesn't throw an exception

https://youtrack.jetbrains.com/issue/KTOR-528

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1437

Ktor 1.2.4 - Client

OkHttp
JSON
Serialization

Describe the bug
It seems that it doesn't catch 404's in an error

To Reproduce

Here's my code:

val httpClientEngine by lazy {
    OkHttp.create {
        if (BuildConfig.DEBUG) {
            addInterceptor(HttpLoggingInterceptor().apply {
                level = HttpLoggingInterceptor.Level.BODY
            })
        }
    }
}

private val httpClient = HttpClient(httpClientEngine) {
    install(JsonFeature) {
        serializer = KotlinxSerializer(Json.nonstrict)
    }
}

suspend fun myFunction(data: Any) = httpCall {
    httpClient.call("my_invalid_url") {
        contentType(ContentType.Application.Json
        method = HttpMethod.Post
        body = MyRequest(data)
    }
}

And here's the httpCall wrapper:

suspend inline fun <T> httpCall(call: () -> T) = try {
    call()
} catch (exception: ClientRequestException) {
    val errorResponse = exception.response.receive<MyErrorResponse>()
    var message = errorResponse.message

    throw Exception(message)
}

However, 404's just pass through this.

Here's the response from the server:

<-- 404 Not Found http://127.0.0.1/my_invalid_url (420ms)        
Server: nginx/1.15.12                                               
Date: Wed, 13 Nov 2019 07:18:25 GMT                                 
Content-Type: text/html; charset=utf-8                              
Content-Length: 152                                                 
Connection: keep-alive                                              
X-Powered-By: Express                                               
Access-Control-Allow-Origin: *                                      
Content-Security-Policy: default-src 'none'                         
X-Content-Type-Options: nosniff                                     
<!DOCTYPE html>                                                     
<html lang="en">                                                    
<head>                                                              
<meta charset="utf-8">                                              
<title>Error</title>                                                
</head>                                                             
<body>                                                              
<pre>Cannot POST /my_invalid_url</pre>                                
</body>                                                             
</html>                                                             
<-- END HTTP (152-byte body)

I'm thinking I should need to register an XML serializer? But I can't find anything in the documentation. If I'm missing something, please do point me in the right direction.

Many thanks!

Expected behavior
It should catch 404's as an error

[Question] CPU 100% Apache Ktor Http Client

ClassCastException launching

https://youtrack.jetbrains.com/issue/KTOR-498

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1540

Ktor Version and Engine Used (client or server and name)
kotlin_version=1.3.61
ktor_version=1.3.0-rc2
coroutines_version=1.3.3-native-mt

Describe the bug
Calling testNetworkCall() from ViewController#viewDidLoad in iOS leads to a kotlin.ClassCastException

  GlobalScope.launch {
    HttpClient { }.get {
      url {
        takeFrom("https://api.basebeta.com")
        encodedPath = "/rankings"
      }
    }
  }

Full stacktrace

Uncaught Kotlin exception: kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for AwaitContinuation(Shareable[used]){HttpResponseData=(statusCode=200 OK)}@cb3ac8. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at 0   BaseBetaCommonApi                   0x0000000111c1ac63 kfun:kotlin.Error.<init>(kotlin.String?;kotlin.Throwable?)kotlin.Error + 115
        at 1   BaseBetaCommonApi                   0x0000000111d20703 kfun:kotlinx.coroutines.CoroutinesInternalError.<init>(kotlin.String;kotlin.Throwable)kotlinx.coroutines.CoroutinesInternalError + 115
        at 2   BaseBetaCommonApi                   0x0000000111d86cba kfun:kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx-coroutines-core(kotlin.Throwable?;kotlin.Throwable?) + 938
        at 3   BaseBetaCommonApi                   0x0000000111d868b0 kfun:kotlinx.coroutines.DispatchedTask.run() + 3568
        at 4   BaseBetaCommonApi                   0x0000000111d1b908 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
        at 5   BaseBetaCommonApi                   0x0000000111da26f1 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
        at 6   BaseBetaCommonApi                   0x0000000111da9d12 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
        at 7   BaseBetaCommonApi                   0x0000000111da9efb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.invoke#internal + 59
        at 8   BaseBetaCommonApi                   0x0000000111da9f5b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.$<bridge-UNN>invoke()#internal + 59
        at 9   BaseBetaCommonApi                   0x0000000111c4fd31 WorkerLaunchpad + 177
        at 10  BaseBetaCommonApi                   0x0000000112052d89 _ZN6Worker19processQueueElementEb + 2569
        at 11  BaseBetaCommonApi                   0x0000000112053336 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 12  libsystem_pthread.dylib             0x00007fff51bfe2eb _pthread_body + 126
        at 13  libsystem_pthread.dylib             0x00007fff51c01249 _pthread_start + 66
        at 14  libsystem_pthread.dylib             0x00007fff51bfd40d thread_start + 13
Caused by: kotlin.ClassCastException: kotlin.coroutines.native.internal.CompletedContinuation cannot be cast to kotlinx.coroutines.DispatchedContinuation
        at 0   BaseBetaCommonApi                   0x0000000111c21227 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87
        at 1   BaseBetaCommonApi                   0x0000000111c1a585 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85
        at 2   BaseBetaCommonApi                   0x0000000111c1a0c5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85
        at 3   BaseBetaCommonApi                   0x0000000111c1ae65 kfun:kotlin.ClassCastException.<init>(kotlin.String?)kotlin.ClassCastException + 85
        at 4   BaseBetaCommonApi                   0x0000000111c6ba5b ThrowClassCastException + 939
        at 5   BaseBetaCommonApi                   0x0000000111d14ef8 kfun:kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(kotlin.coroutines.Continuation<#STAR>) + 216
        at 6   BaseBetaCommonApi                   0x0000000111c42400 kfun:kotlin.coroutines.native.internal.ContinuationImpl.releaseIntercepted() + 768
        at 7   BaseBetaCommonApi                   0x0000000111c42a62 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1234
        at 8   BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 9   BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 10  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 11  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 12  BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 13  BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 14  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 15  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 16  BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 17  BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 18  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 19  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 20  BaseBetaCommonApi                   0x0000000111d8659c kfun:kotlinx.coroutines.DispatchedTask.run() + 2780
        at 21  BaseBetaCommonApi                   0x0000000111d1b908 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
        at 22  BaseBetaCommonApi                   0x0000000111da26f1 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
        at 23  BaseBetaCommonApi                   0x0000000111da9d12 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
        at 24  BaseBetaCommonApi                   0x0000000111da9efb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.invoke#internal + 59
        at 25  BaseBetaCommonApi                   0x0000000111da9f5b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.$<bridge-UNN>invoke()#internal + 59
        at 26  BaseBetaCommonApi                   0x0000000111c4fd31 WorkerLaunchpad + 177
        at 27  BaseBetaCommonApi                   0x0000000112052d89 _ZN6Worker19processQueueElementEb + 2569
        at 28  BaseBetaCommonApi                   0x0000000112053336 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 29  libsystem_pthread.dylib             0x00007fff51bfe2eb _pthread_body + 126

To Reproduce
Copy and paste the code snippet from above into the common module of a multiplatform project. Run the function from ViewController#viewDidLoad in the iOS app.

Expected behavior
I expect the code to execute without throwing a ClassCastException.

Ktor HttpClient unable to download files larger than 4096 bytes

https://youtrack.jetbrains.com/issue/KTOR-475

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1636

Ktor Version and Engine Used (client or server and name)
Ktor Client 1.3.1 in Kotlin MPP context. Engines OkHTTP and iOS Native, but not explicitly set (resolved by gradle dependency).

Describe the bug
Ktor HttpClient keeps suspending when requesting to read more than 4096 bytes from a Url, and never comes back.

To Reproduce
Steps to reproduce the behavior:

suspend fun downloadFile(url: String): ByteArray {
    val httpClient = HttpClient{
        install(Logging) {
            this.logger = Logger.SIMPLE
            this.level = LogLevel.ALL
        }
    }
    val statement = httpClient.request<HttpStatement>(url) {
        range?.let {
            header(HttpHeaders.Range, range)
        }
    }
    return statement.execute {
        val contentLength = it.contentLength()?.lowInt ?: 0
        val byteArray = ByteArray(contentLength)
        var offset = 0
        do {
            val currentRead = it.content.readAvailable(byteArray, offset, byteArray.size)
            offset += currentRead
            logger.logDebug("Download in progress, offset: ${offset}, current read ${currentRead} / ${contentLength}")
        } while (offset < contentLength)
        logger.logDebug("Download done")
        return@execute byteArray
    }
}

Execute function above with a URL that contains a file larger than 4096 bytes. During my testing I'm downloading files from Amazon S3.

Expected behavior
Expected to get a byte array containing the entire contents of the file.

Screenshots
The logging in the function above generates this result:

Download in progress, offset: 4088, current read 4088 / 355835
Download in progress, offset: 4096, current read 8 / 355835

I know that using a ByteArray is not the most ideal solution to download a file into, but I'm working in a pure-kotlin Multi Platform Project, so any Java import is a no-go without using expect/actual which means I cannot directly write to files, or use a stream of some kind.

Can't compile project with Ktor JS client because of webpack errors

Client sometimes doesn't do response validation

Unable to stream http response - readUTF8Line returns null continuously

Internal Compilation Error when Deconstructing in formData

Ktor client catch ConnectException without printing stacktrace

Disable system out for dropped connections

Cookies per request

Reuse Serialization from Server in Client

JSON feature fails with io.ktor.client.features.ClientRequestException: Client request invalid: 406 Not Acceptable

https://youtrack.jetbrains.com/issue/KTOR-357

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1907

Ktor Version and Engine Used (client or server and name)

implementation "io.ktor:ktor-client-cio:$ktor_version"
implementation "io.ktor:ktor-client-json-jvm:$ktor_version"
implementation "io.ktor:ktor-client-gson:$ktor_version"
ext.ktor_version = "1.3.1"

JVM 1.8, with Kotlin v1.3.72

Describe the bug
With JSON support being enabled, it can not consume a REST API.

Example Implementation

import io.ktor.client.HttpClient
import io.ktor.client.features.json.JsonFeature
import io.ktor.client.request.get


suspend fun main() {
    val clusterJson: String = HttpClient().get("http://ubuntu:8082/v3/clusters")
    println(clusterJson)

    val client: V3Cluster = HttpClient() {
        install(JsonFeature)
    }.get("http://ubuntu:8082/v3/clusters")

}

data class V3Cluster(
        val `data`: List<Data>
)

data class Data(
        val attributes: Attributes,
        val id: String,
        val type: String
)

data class Attributes(
        val cluster_id: String
)

This fails with

Exception in thread "main" io.ktor.client.features.ClientRequestException: Client request(http://ubuntu:8082/v3/clusters) invalid: 406 Not Acceptable
	at io.ktor.client.features.DefaultResponseValidationKt$addDefaultResponseValidation$1$1.invokeSuspend(DefaultResponseValidation.kt:35)
	at io.ktor.client.features.DefaultResponseValidationKt$addDefaultResponseValidation$1$1.invoke(DefaultResponseValidation.kt)
	at io.ktor.client.features.HttpCallValidator.validateResponse(HttpCallValidator.kt:35)
	at io.ktor.client.features.HttpCallValidator$Companion$install$2.invokeSuspend(HttpCallValidator.kt:96)
	at io.ktor.client.features.HttpCallValidator$Companion$install$2.invoke(HttpCallValidator.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:273)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.client.call.HttpClientCall.receive(HttpClientCall.kt:75)
	at controllers.KafkaClusterKt.main(KafkaCluster.kt:105)
....

The first get request is processed correctly, but the second one fails with the provided stacktrace.

Expected behaviour
It should deserialize the json response into an object.

Without JSON feature, the request works fine and is producing the JSON as output. Also when using curl on the terminal, the API endpoint itself seems fine:

brandl@ubuntu:~$ curl http://ubuntu:8082/v3/clusters | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   594  100   594    0     0  84857      0 --:--:-- --:--:-- --:--:-- 84857
{
  "links": {
    "self": "http://rest-proxy:8082/v3/clusters",
    "next": null
  },
  "data": [
    {
      "id": "crn:///kafka=CVyJNKWhSrmXmnRUKWo5cg",
      "links": {
        "self": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg"
      },
      "attributes": {
        "cluster_id": "CVyJNKWhSrmXmnRUKWo5cg"
      },
      "relationships": {
        "controller": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/brokers/1"
          }
        },
        "brokers": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/brokers"
          }
        },
        "topics": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/topics"
          }
        }
      },
      "type": "KafkaCluster"
    }
  ]
}

Not relevant for the ticket, but maybe some context is helpful: I'm trying to cusome the confluent broker config API v3 as described under https://github.com/confluentinc/kafka-rest/issues/676

ktor exception of unknown origin crashing my app

https://youtrack.jetbrains.com/issue/KTOR-1039

I get these exceptions in the log with ktor 1.3.0, but with 1.3.2 it crashes my app:

2020-09-16 10:39:40.942 30696-30937/com.thomashabets.simapprover I/Process: Sending signal. PID: 30696 SIG: 9

I'm using WebSockets and some URL fetching, nothing fancier than that.

I'm unclear on what's going on here. I'm new to kotlin, and to app development, so it could be that I'm doing something wrong. But since nothing in the stack trace is mine, I don't know where to start. It's possible that there's a race condition I'm failing to protect against.

Or is this a bug in ktor? Because exceptions in ktor code, that don't seem catchable, and this different behavior between 1.3.0 and 1.3.2, I'm suspecting it might be, and therefore am creating this ticket.

My ugly source code is here.

For now I'm staying with 1.3.0 because at least it doesn't crash my app, but only manifests in log pollution.

2020-09-16 10:39:40.941 30696-30948/com.thomashabets.simapprover E/AndroidRuntime: FATAL EXCEPTION: ktor-cio-dispatcher-worker-3
    Process: com.thomashabets.simapprover, PID: 30696
    java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:421)
        at io.ktor.utils.io.nio.ChannelsKt.read(Channels.kt:135)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invokeSuspend(CIOReader.kt:90)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invoke(Unknown Source:10)
        at io.ktor.network.util.UtilsKt.withSocketTimeout(Utils.kt:20)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:80)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:175)
        at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:137)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:306)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:316)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:248)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:84)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:64)
        at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:73)
        at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(Unknown Source:12)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
2020-09-16 10:39:40.942 30696-30980/com.thomashabets.simapprover E/AndroidRuntime: FATAL EXCEPTION: ktor-cio-dispatcher-worker-3
    Process: com.thomashabets.simapprover, PID: 30696
    java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:421)
        at io.ktor.utils.io.nio.ChannelsKt.read(Channels.kt:135)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invokeSuspend(CIOReader.kt:90)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invoke(Unknown Source:10)
        at io.ktor.network.util.UtilsKt.withSocketTimeout(Utils.kt:20)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:80)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:175)
        at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:137)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:306)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:316)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:248)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:84)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:64)
        at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:73)
        at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(Unknown Source:12)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

WebSocket throws a vague IOException when internet is disconnected

https://youtrack.jetbrains.com/issue/KTOR-623

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1119

Ktor Version

1.2.0-rc

Ktor Engine Used(client or server and name)

Client/CIO

JVM Version, Operating System and Relevant Context

OpenJDK 11.0.3, Arch Linux.

Feedback

When the internet is disconnected, a vague java.io.IOException is thrown twice, and it cannot be caught.

Exception in thread "pool-2-thread-3" java.io.IOException: Connection reset by peer
	at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
	at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
	at kotlinx.io.nio.ChannelsKt.read(Channels.kt:117)
	at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:75)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.ResumeModeKt.resumeMode(ResumeMode.kt:67)
	at kotlinx.coroutines.DispatchedKt.resume(Dispatched.kt:309)
	at kotlinx.coroutines.DispatchedKt.resumeUnconfined(Dispatched.kt:49)
	at kotlinx.coroutines.DispatchedKt.dispatch(Dispatched.kt:295)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:250)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:260)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:189)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:83)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:63)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:74)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Add ability to override HttpAsyncResponseConsumer

HttpClient cancels parent coroutine despite try-catch block

JS client: Fail to fetch in browser

Allow to create threadless engine (CIO) for Ktor client

Can not find module ktor-ktor-http in npm.

https://youtrack.jetbrains.com/issue/KTOR-595

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1224

First I would very appreciate for the ktor framework, but I have a little problem of my usage. I compile the kotlin code to JS. but in the generated JS code it contain this require("ktor-ktor-http") code. So I searched ktor-ktor-http in the npm repositories, but unfortunately I can not find it in the the npm repositories. So, did I missed something?

this is my generated JS code


function (root, factory) {
  if (typeof define === 'function' && define.amd)
    define(['exports', 'kotlin', 'kotlinx-coroutines-core', 'ktor-ktor-client-core', 'ktor-ktor-http'], factory);
  else if (typeof exports === 'object')
    factory(module.exports, require('kotlin'), require('kotlinx-coroutines-core'), require('ktor-ktor-client-core'), require('ktor-ktor-http'));
  else {
    if (typeof kotlin === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'kotlin' was not found. Please, check whether 'kotlin' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['kotlinx-coroutines-core'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'kotlinx-coroutines-core' was not found. Please, check whether 'kotlinx-coroutines-core' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['ktor-ktor-client-core'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'ktor-ktor-client-core' was not found. Please, check whether 'ktor-ktor-client-core' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['ktor-ktor-http'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'ktor-ktor-http' was not found. Please, check whether 'ktor-ktor-http' is loaded prior to 'kb-mutiplatform'.");
    }
    root['kb-mutiplatform'] = factory(typeof this['kb-mutiplatform'] === 'undefined' ? {} : this['kb-mutiplatform'], kotlin, this['kotlinx-coroutines-core'], this['ktor-ktor-client-core'], this['ktor-ktor-http']);
  }

this is my build.gradle


plugins {
    id 'kotlin-multiplatform' version '1.3.40'
}
repositories {
    maven {
        url 'http://maven.aliyun.com/nexus/content/groups/public/'
    }
    maven {
        url 'http://maven.aliyun.com/nexus/content/repositories/jcenter'
    }
    mavenLocal()
    mavenCentral()

    maven { url 'https://plugins.gradle.org/m2/' }
    maven { url 'https://dl.bintray.com/kotlin/kotlin-eap' }
    jcenter()
}
group 'com.example'
version '0.0.1'

apply plugin: 'maven-publish'

kotlin {
    jvm()
    js {
        compilations.main.kotlinOptions.moduleKind = 'umd'
    }
    // For ARM, should be changed to iosArm32 or iosArm64
    // For Linux, should be changed to e.g. linuxX64
    // For MacOS, should be changed to e.g. macosX64
    // For Windows, should be changed to e.g. mingwX64
//    macosX64("macos")
    iosX64("ios") {
        binaries {
            framework()
        }
    }
    sourceSets {
        commonMain {
            dependencies {
                implementation ("io.ktor:ktor-client-core:$ktor_version")
                implementation kotlin('stdlib-common')
            }
        }
        commonTest {
            dependencies {
                implementation kotlin('test-common')
                implementation kotlin('test-annotations-common')
            }
        }
        jvmMain {
            dependencies {
                implementation ("io.ktor:ktor-client-android:$ktor_version")
                implementation kotlin('stdlib-jdk8')
            }
        }
        jvmTest {
            dependencies {
                implementation kotlin('test')
                implementation kotlin('test-junit')
            }
        }
        jsMain {
            dependencies {
                implementation kotlin('stdlib-js')
                implementation "io.ktor:ktor-client-js:$ktor_version"
            }
        }
        jsTest {
            dependencies {
                implementation kotlin('test-js')
            }
        }
        iosMain {
            dependencies {
                implementation ("io.ktor:ktor-client-ios:$ktor_version")
            }
        }
        iosTest {
        }
    }
}

this is my kotlin code



class LoginApi {
    private val client = HttpClient()

    fun login(param: LoginReq, callback: (String) -> Unit) {
        GlobalScope.apply {
            launch(ApplicationDispatcher) {
                val result: String = client.post {
                    url("http://10.76.34.177:3000/user/login")
                    parameter("userName", param.userName)
                    parameter("passWord", param.passWord)
                }

                callback(result)
            }
        }
    }
}

I'm very newly in kotlin, So I will very appreciate of you reply.

Core

Netty HTTP/2 not working

https://youtrack.jetbrains.com/issue/KTOR-578

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1261

The problem reproduces with http2-push from https://github.com/ktorio/ktor-samples

Run ./gradlew :http2-push:run

const val kotlin = "1.3.41"
const val ktor = "1.2.3"

give a exception

Exception in thread "main" java.lang.NoClassDefFoundError: io.netty.handler.ssl.OpenSsl (initialization failure)
	at java.base/java.lang.J9VMInternals.initializationAlreadyFailed(J9VMInternals.java:143)
	at io.netty.handler.ssl.SslContext.defaultProvider(SslContext.java:120)
	at io.netty.handler.ssl.SslContext.defaultServerProvider(SslContext.java:107)
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:454)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:457)
	at io.ktor.server.netty.NettyChannelInitializer.<init>(NettyChannelInitializer.kt:72)
	at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:103)
	at io.ktor.server.netty.Netty.create(Embedded.kt:14)
	at io.ktor.server.netty.Netty.create(Embedded.kt:12)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:79)
	at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:77)
	at io.ktor.samples.http2push.MainKt.main(Main.kt:17)
Caused by: java.lang.NoSuchMethodError: io/netty/internal/tcnative/SSLContext.setCipherSuite(JLjava/lang/String;Z)Z (loaded from file:/Users/icode/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative/2.0.7.Final/b9413ea71da2d14e6f5644aa33a5c4d966090540/netty-tcnative-2.0.7.Final.jar by jdk.internal.loader.ClassLoaders$AppClassLoader@760c26b8) called from class io.netty.handler.ssl.OpenSsl (loaded from file:/Users/icode/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.37.Final/a242f98517b4dc1a53a0eb7dad8ad30d4f4b9f2e/netty-handler-4.1.37.Final.jar by jdk.internal.loader.ClassLoaders$AppClassLoader@760c26b8).
	at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:201)
	at io.ktor.server.netty.NettyChannelInitializer$Companion.findAlpnProvider(NettyChannelInitializer.kt:158)
	at io.ktor.server.netty.NettyChannelInitializer$Companion.access$findAlpnProvider(NettyChannelInitializer.kt:147)
	at io.ktor.server.netty.NettyChannelInitializer$Companion$alpnProvider$2.invoke(NettyChannelInitializer.kt:148)
	at io.ktor.server.netty.NettyChannelInitializer$Companion$alpnProvider$2.invoke(NettyChannelInitializer.kt:147)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at io.ktor.server.netty.NettyChannelInitializer$Companion.getAlpnProvider$ktor_server_netty(NettyChannelInitializer.kt)
	at io.ktor.server.netty.NettyChannelInitializer.<init>(NettyChannelInitializer.kt:58)
	... 6 more

I update tcnative_version to 2.0.25.Final , then start is ok
I get this exception when request
Navigate to https://localhost:8443/

2019-08-02 20:23:13.309 [nioEventLoopGroup-3-2] WARN  i.n.h.s.ApplicationProtocolNegotiationHandler - [id: 0x00f04f96, L:/0:0:0:0:0:0:0:1:443 - R:/0:0:0:0:0:0:0:1:53289] TLS handshake failed:
javax.net.ssl.SSLHandshakeException: error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.shutdownWithError(ReferenceCountedOpenSslEngine.java:965)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1231)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1185)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1256)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1299)
	at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:204)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1236)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1273)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:500)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:439)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
2019-08-02 20:23:13.310 [nioEventLoopGroup-3-2] WARN  i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:470)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.shutdownWithError(ReferenceCountedOpenSslEngine.java:965)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1231)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1185)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1256)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1299)
	at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:204)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1236)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1273)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:500)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:439)
	... 17 common frames omitted

Support for Unix domain sockets for both server and client

The beginning character of endodedPath field(Url class) is wrong when relative path

consumeEachBufferRange completes successfully on connection abort

Invalid response without error

Invalid HTTP version should fail

The colon after the host parameter requires a port

Allow for extensible frame types in accordance with RFC-6455 Section 5.8

Allow resolving URLs against another URL similar to a browser

Add modulepath support for Java >= 9 breaking-change

Request may not include Host parameter

Host parameter may contain illegal symbols

parameterOf() should have a variant that takes in a Map<String, List<String>>

Incorrect parsing of query parameters

Better interceptor chain on errors

https://youtrack.jetbrains.com/issue/KTOR-607

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1186

In case of any errors, either a faulty route which throws an exception or requests to a non-existing route, the interceptor chain behaves differently than on requests to routes which responds properly.

In a normal situation the ApplicationSendPipeline is executed during the execution of the ApplicationCallPipeline, which enables us to trace an entire request/response situation using e.g. the Monitoring phase:

intercept(ApplicationCallPipeline.Monitoring) {
     try {
         log.info("incoming request ${call.request.httpMethod.value} ${call.request.uri}")
         proceed()
     } catch (err: Throwable) {
         log.info(err.message, err)
         throw err
     } finally {
         log.info("responding with ${call.response.status()}")
     }
}

Because we are executing within the ApplicationCallPipeline we get the added bonus that our log messages gets enriched with stuff like CallId.

However, when an error occurs (because the route does not exist, or the handler threw an exception) the ApplicationCallPipeline gets interrupted. Then some error handling is done by the DefaultEnginePipeline which tries to set a reasonable status code (404 in case of a non-existing route, 500 in case of exceptions) and logs the exception (no CallId here!).
But because the ApplicationCallPipeline got interrupted, the Monitoring interceptor from above has already exited, so it did never get a chance to act on the response code set by DefaultEnginePipeline.

After DefaultEnginePipeline has done its thing, the ApplicationSendPipeline is executed with the new response. And because ApplicationSendPipeline does not share any attributes with the ApplicationCallPipeline, we do not get the bonus of having things like CallId.

In short, it's hard to properly log a request with the resulting status code, with things like CallId. Because if we want to handle the quirks I've just mentioned, we have to do something like:

intercept(ApplicationCallPipeline.Monitoring) {
     try {
         log.info("incoming request ${call.request.httpMethod.value} ${call.request.uri}")
         proceed()
     } catch (err: Throwable) {
         log.info(err.message, err)
         throw err
     }
}

// ensure that the callId is present during the whole ApplicationSendPipeline execution
sendPipeline.intercept(ApplicationSendPipeline.Before) {
    try {
        MDC.put("call_id", call.callId)
        proceed()
    } finally {
        MDC.remove("call_id")
    }
}

sendPipeline.intercept(ApplicationSendPipeline.After) { message ->
    (when (message) {
        is OutgoingContent -> message.status
        is HttpStatusCode -> message
        else -> null
    } ?: context.response.status())?.let { status ->
        log.info("responding with ${status.value}")
    } ?: log.info("sending unknown response code")
}

socket.isClosed remains `false` after socket.close() call

https://youtrack.jetbrains.com/issue/KTOR-706

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/833

Hi, I'm using io.ktor:ktor-network-tls:1.0.1[1], running JDK8 on MacOSX 10.10.5
The minimal working example to reproduce the problem is as follows:

import io.ktor.network.selector.ActorSelectorManager
import io.ktor.network.sockets.aSocket
import io.ktor.network.sockets.isClosed
import io.ktor.network.sockets.openReadChannel
import io.ktor.network.sockets.openWriteChannel
import kotlinx.coroutines.Dispatchers.Unconfined
import kotlinx.coroutines.newSingleThreadContext
import kotlinx.coroutines.runBlocking

/** Created by revin(caoyunbin@baidu.com) on 2018-12-26. */
fun main(args:Array<String>)=runBlocking(Unconfined){
    val tcp=aSocket(ActorSelectorManager(newSingleThreadContext("httpIO"))).tcp()
    val socket=tcp.connect("www.baidu.com",80)
    socket.openReadChannel();socket.openWriteChannel()
    socket.close();println(socket.isClosed)
}

The result expected from the above code is true but instead I got false

I can't use io.ktor:ktor-network-tls:1.1.0, which is the latest version as of writing. The following error shows up when trying to build the project. So I'm using an earlier version to report the problem

CONFIGURE FAILED in 7s
Could not find org.jetbrains.kotlinx:atomicfu:0.12.0.
Searched in the following locations:
    https://repo1.maven.org/maven2/org/jetbrains/kotlinx/atomicfu/0.12.0/atomicfu-0.12.0.pom
    https://repo1.maven.org/maven2/org/jetbrains/kotlinx/atomicfu/0.12.0/atomicfu-0.12.0.jar
Required by:
    project :jvm > io.ktor:ktor-network-tls:1.1.0
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > org.jetbrains.kotlinx:kotlinx-io-jvm:0.1.2
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > org.jetbrains.kotlinx:kotlinx-coroutines-io-jvm:0.1.2
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > io.ktor:ktor-network:1.1.0

According to maven(https://mvnrepository.com/artifact/io.ktor/ktor-network-tls/1.1.0), ktor-network-tls relies on ktor-http-cio-native, is that intentional?

ktor-network - client socket close throws exception on server that, even if caught, prints stack trace in console

https://youtrack.jetbrains.com/issue/KTOR-459

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1686

Ktor Version and Engine Used (client or server and name)
ktor-network 1.3.1

Describe the bug
Using the slightly modified Socket Echo Server sample code from the ktor docs:

fun main() {
    runBlocking {
        val server = aSocket(ActorSelectorManager(Dispatchers.IO)).tcp().bind(InetSocketAddress("127.0.0.1", 2323))
        println("Started echo telnet server at ${server.localAddress}")

        while (true) {
            val socket = server.accept()

            launch {
                println("Socket accepted: ${socket.remoteAddress}")

                val input = socket.openReadChannel()
                val output = socket.openWriteChannel(autoFlush = true)

                try {
                    while (true) {
                        val line = input.readUTF8Line()

                        println("${socket.remoteAddress}: $line")
                        output.write("$line\r\n")
                    }
                } catch (e: Exception) {
                    // don't print stack trace so we can differentiate
                    //e.printStacktrace()
                    println("caught exception: ${e.message}")
                    socket.dispose()
                }
            }
        }
    }
}

If a client connects and at some point the connection is closed by the client, an IOException is thrown, and caught, but the stacktrace is still printed in the console. It's important to note that the echo server continues running and still accepts new connections.

This does not seem to be a coroutines issue since the following code catches the exception, but no extra stack trace is printed in the console:

fun main() {
    runBlocking {
        launch {
            try {
                throw IOException("test")
            } catch (e: Exception) {
                println("caught exception: ${e.message}")
            }
        }
    }
}

To Reproduce
Using the above server code:

start the server
connect to it (eg telnet localhost 2323)
close the connection (eg close the terminal window)

Expected behavior
Since the exception is caught, it is expected that the stack trace is not printed, since printing the stack trace is not configured anywhere explicitly.

Screenshots
Echo server output:
Снимок экрана 2020-02-28 в 4 49 45 PM

Coroutine only output:
Снимок экрана 2020-02-28 в 4 56 43 PM

Blocking adapters vs single-threaded dispatcher

ktor-html-builder: exceptions in DSL are swallowed when StatusPages feature is installed

Query of pre-signed URL has been altered after decode and re-encode process breaking-change

Session cookie with very long max age duration

Auto push styleLink and others when using ApplicationCall.respondHtml

Is Ktor really that slow compared to other micro java frameworks?

Docs

Timeout documentation is inaccurate

https://youtrack.jetbrains.com/issue/KTOR-271

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/268

https://ktor.io/clients/http-client/features/timeout.html

The Client Timeout Feature states that "By default all these timeouts are infinite"; this doesn't seem to be the case with either the Apache or CIO engines where unless I explicitly set a timeout I get:

Apache

io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=http://localhost:1080/api/import/tickets/v1/file, socket_timeout=unknown] ms
	at io.ktor.client.features.HttpTimeoutKt.SocketTimeoutException(HttpTimeout.kt:170)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt$sendRequest$$inlined$suspendCancellableCoroutine$lambda$2.failed(ApacheHttpRequest.kt:44)
	at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
	at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.SocketTimeoutException: 10,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
	... 11 common frames omitted

CIO (which doesn't seem to accept specifying timeouts at all)

kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 15000 ms
	at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:149)
	at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:119)
	at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:493)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:272)
	at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
	at java.base/java.lang.Thread.run(Thread.java:834)

Mention about closing ActorSelector manager

Provide better support for Ktor clients

parseUrlEncodedParameters Documentation

Can I create a SOAP service with Ktor?

CookieConfiguration should default to secure configuration and require user opt-out

Docs section about testing with cookies is outdated

HTTP/2 documentation outdated

Auth Feature Code Snippet: form authentication the doesn't work

https://youtrack.jetbrains.com/issue/KTOR-821

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/211

I've the below code, route "/" and "/bye" are working fine, but route "login" given blank page once!!

package blog

import kotlinx.html.*
import kotlinx.html.stream.*    // for createHTML
import org.jetbrains.ktor.application.*
import org.jetbrains.ktor.auth.*
import org.jetbrains.ktor.features.*
import org.jetbrains.ktor.http.*
import org.jetbrains.ktor.response.*
import org.jetbrains.ktor.routing.*

import org.jetbrains.ktor.request.*   // for request.uri

import org.jetbrains.ktor.html.*
import org.jetbrains.ktor.pipeline.*

import org.jetbrains.ktor.host.*   // for embededServer
import org.jetbrains.ktor.netty.*  // for Netty

fun main(args: Array<String>) {
    embeddedServer(Netty, 8080, watchPaths = listOf("BlogAppKt"), module = Application::module).start()
}

fun Application.module() {
    install(DefaultHeaders)
    install(CallLogging)

    intercept(ApplicationCallPipeline.Call) { 
        if (call.request.uri == "/hi")
            call.respondText("Test String")
    }

    install(Routing) {
        get("/") {
            call.respondText("""Hello, world!<br><a href="/bye">Say bye?</a>""", ContentType.Text.Html)
        }
        get("/bye") {
            call.respondText("""Good bye! <br><a href="/login">Login?</a> """, ContentType.Text.Html)
        }
        route("/login") {
            authentication {
                formAuthentication { up: UserPasswordCredential ->
                    when {
                        up.password == "ppp" -> UserIdPrincipal(up.name)
                        else -> null
                    }
                }
            }
         
            handle {
                val principal = call.authentication.principal<UserIdPrincipal>()
                if (principal != null) {
                    call.respondText("Hello, ${principal.name}")
                } else {
                        val html = createHTML().html {
                        body {
                            form(action = "/login", encType = FormEncType.applicationXWwwFormUrlEncoded, method = FormMethod.post) {
                                p {
                                    +"user:"
                                    textInput(name = "user") {
                                        value = principal?.name ?: ""
                                    }
                                }

                                p {
                                    +"password:"
                                    passwordInput(name = "pass")
                                }

                                p {
                                    submitInput() { value = "Login" }
                                }
                            }
                        }
                    }
                    call.respondText(html, ContentType.Text.Html)
                }
            }
        }
    }
}

Initializing KotlinxSerializer to handle lists is needlessly confusing

`getDigestFunction` method with constant salt is not supported after ktor 1.2.0

New Mock Client Does not allow access to the properties of a request

example using cookies with JS client

Ktor on Google App Engine flexible environment

[Outdated doc] Ktor quickstart contains outdated reference to version before 1.0

start.ktor.io - Incorrect import for websockets for ktor 1.2.4

Client logging docs don't mention all required dependencies

Out of date self-signed-certificate documentation

Custom JSON mapping with Jackson

Documentation for kotlinx.serialization wrong and outdated

Prometheus Registry unavailable for Micrometer

metrics prometheus better documentation

Use hugo for building website?

Add previous and next navigation links to bottom of pages

https://youtrack.jetbrains.com/issue/KTOR-287

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/105

Problem

As a new Ktor user I wanted to read the entire manual after getting started with the "Quick Start" section and before doing any serious coding.

It got frustrating reading the "Servers" section because once I'd reached the end of a long page, I'd have to scroll back towards the top, find the active navigation link, and then click on the next one.

Solution

Add previous and next links at the bottom of each page that has a category. I have a local branch that implements this approach that I could merge and use to submit a PR after I finish it.

Update

I have a working branch here: https://github.com/kensiprell/ktorio.github.io/tree/prev-next. I don't want to create a PR until someone takes a look because it touches so much.

_layouts/page.html:

        {% if page.status == 'todo' %}
          Coming soon...
        {% else %}
          {{ content }}
        {% endif %}

        {% if page.category != null %}
            {% include prev-next.html %}
        {% endif %}
      </div>
    </div>
  </div>
</div>

_includes/prev-next.html:

{% assign entries = site.pages | sort: "path" %}
{% for entry in entries %}
    {% if entry.url == page.url %}
        {% assign prevIndex = forloop.index0 | minus: 1 %}
        {% assign nextIndex = forloop.index0 | plus: 1 %}
        <div class="container-fluid col-sm-12" style="padding-top:2rem;border-top:1px solid #ddd;">
            <div class="row">
                <div class="col-sm-6">
                    {% if prevIndex %}
                        {% if entries[prevIndex].category == page.category %}
                            <a href="{{ entries[prevIndex].url }}">
                                &larr; {{ entries[prevIndex].title }}
                            </a>
                        {% else %}
                            First page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
                <div class="col-sm-6 text-right">
                    {% if nextIndex %}
                        {% if entries[nextIndex].category == page.category %}
                            <a href="{{ entries[nextIndex].url }}">
                                {{ entries[nextIndex].title }} &rarr;
                            </a>
                        {% else %}
                            Last page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
            </div>
        </div>
    {% endif %}
{% endfor %}

Here are some images showing the proof of concept.

First page in the "Kotlinx" category:

prev-next-1

Second page in the "Kotlinx" category:

prev-next-2

Last page in the "Kotlinx" category:

prev-next-3

The real work involves making children directory names match the file names and then adding permalinks (see image below). I used Jekyll for the first time two days ago, so I'm not 100% certain that my approach won't have negative ramifications elsewhere. However, I haven't noticed any issues running it locally on my computer.

prev-next

What is a "VOs"

SSLContext on CIO?

Maven examples are incorrect for ktor 1.2.0

Circular Reference

Add links to complete examples (github/zip) on every quickstart sub-page

Update documentation for FatJar generation in ktor.io

https://youtrack.jetbrains.com/issue/KTOR-656

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1004

Ktor Version

1.1.3 with Tomcat and Gradle

Operating System

Operating system: MacOS High Sierra

Feedback

By following the steps in documentation to generate a FatJar, whenever I try to execute my Ktor application by using the command java -jar pathToFatJar, I receive de following error:

Exception in thread "main" java.lang.UnsupportedOperationException: Packages and file facades are not yet supported in Kotlin reflection. Meanwhile please use Java reflection to inspect this class: class com.tidelevel.ApplicationKt
	at kotlin.reflect.jvm.internal.KClassImpl.reportUnresolvedClass(KClassImpl.kt:301)
	at kotlin.reflect.jvm.internal.KClassImpl.access$reportUnresolvedClass(KClassImpl.kt:43)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$descriptor$2.invoke(KClassImpl.kt:53)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$descriptor$2.invoke(KClassImpl.kt:44)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:92)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:31)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getDescriptor(KClassImpl.kt)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$objectInstance$2.invoke(KClassImpl.kt:106)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazyVal.invoke(ReflectProperties.java:62)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:31)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getObjectInstance(KClassImpl.kt)
	at kotlin.reflect.jvm.internal.KClassImpl.getObjectInstance(KClassImpl.kt:239)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createModuleContainer(ApplicationEngineEnvironmentReloading.kt:328)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:317)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:275)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:127)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:247)
	at io.ktor.server.tomcat.TomcatApplicationEngine.start(TomcatApplicationEngine.kt:108)
	at io.ktor.server.tomcat.EngineMain.main(EngineMain.kt:16)

This error just happen when trying to execute the FatJar in command line, everything works fine when running in Intellij. I have seen similar issues at:

By changing the mainClassName property to correspond to my own Application class, everything seems to work fine! I am not sure if the documentation needs to be updated in this page or if I did something wrong.

Hope I receive an answer from you!

Fat JAR documentation lacks information

Add support for the SameSite cookie attribute

"Using a Self-Signed Certificate" docs provide wrong dependency for 1.3.x

Generator

Ktor init tools: Unsupported: 'Float'

Infrastructure

client web-socket does not connect on graal native-image on Windows

ktor-server-test-host indirectly depends on logback-classic

Integration with detekt-hint

empty io.ktor:ktor-client-cio:1.4.0 on central repo

Process

Ktor 1.3.2 missing from Maven Central

Samples

Routing: get matcher has higher priority than param matcher on the same level

Add Unit Tests

Tomcat war does not work properly on a tomcat server

SDK location not found while creating sample jetty-embedded

Revert kodein sample

When I try to start fullstack-mpp, No such property: DefaultKotlinCompilationOutput

How run ktor app in tomcat in diferent path?

[deployments/docker] Show using GoogleContainerTools/jib

androidNative

Self-contained docker build (improvement)

Can not build AppEngine sample

Add auto-reload sample?

Cannot add task 'run' as a task with that name already exists.

Proguard on Android issues

[SECURITY] 0.9.* Releases were built/executed/released in the context of insecure/untrusted code

Please remove or update these samples, because they do not compile with new versions of Kotlin

Form authorization not working in example (feature/auth)

Need help compiling fullstack-mpp.

Missing dependency io.ktor:ktor-client-ios_debug_ios_arm64:1.1.1

ApplicationCall.push fails under HTTP/1.1

Corrupted files with low connection rate (Jetty)

Unable To Run TimerTask Continuously From Application.main Method

https://youtrack.jetbrains.com/issue/KTOR-190

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/31

Expected

I'm attempting to have my Ktor app's main method run automatically once deployed to AppEngine, similar to how a Jars main method runs when uploaded to AppEngine.

Actual

Currently the Ktor application's main method is only called when the app's hosted route is called: https://[yourProjectName].appspot.com.

In the applications's main method is logic to retrieve content from an API request based on a Timer Task and save that information to a Firestore database which a client consumes. The goal is to have this task running continuously.

This logic currently works when deployed in a Jar to AppEngine. However, implementing this with Ktor would save deploy time and help future proof the backend service if endpoints are required.

Setup

Main Method

import io.ktor.application.Application
fun Application.main() {
    // App logic here.
   Timer().scheduleAtFixedRate(_TaskName_()), 0, 60000) 
   ...
}

build.gradle

buildscript {
ext.kotlin_version = '1.3.10'
ext.ktor_version = '1.0.0'
ext.appengine_version = '1.9.60'
ext.appengine_plugin_version = '1.3.4'
ext.junitJupiterVersion  = '5.0.3'

repositories {
    jcenter()
    mavenCentral()
}
dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
    classpath "com.google.cloud.tools:appengine-gradle-plugin:$appengine_plugin_version"
    classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.3'
    }
}

plugins {
    id 'org.jetbrains.kotlin.jvm' version '1.2.51'
}

apply plugin: 'java'
apply plugin: 'kotlin'
apply plugin: 'war'
apply plugin: 'com.google.cloud.tools.appengine'

sourceSets {
    main.kotlin.srcDirs = [ 'src/main/kotlin' ]
}

sourceCompatibility = 1.8

repositories {
    jcenter()
    mavenCentral()
    maven { url "https://kotlin.bintray.com/ktor" }
}

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
    implementation "io.ktor:ktor-server-servlet:$ktor_version"
    implementation "io.ktor:ktor-html-builder:$ktor_version"
    providedCompile "com.google.appengine:appengine:$appengine_version"
    implementation 'com.squareup.retrofit2:retrofit:2.3.0'
    implementation 'com.squareup.retrofit2:converter-gson:2.3.0'
    implementation 'com.squareup.retrofit2:adapter-rxjava:2.3.0'
    implementation 'io.reactivex.rxjava2:rxjava:2.1.1'
    implementation 'com.google.firebase:firebase-admin:6.3.0'
    implementation 'com.google.apis:google-api-services-youtube:v3-rev204-1.23.0'
    testCompile group: 'junit', name: 'junit', version: '4.12'
    // JUnit Jupiter API and TestEngine implementation
    testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")
    testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")
    testCompile("org.assertj:assertj-core:3.10.0")
    // To avoid compiler warnings about @API annotations in JUnit code
    testCompileOnly('org.apiguardian:apiguardian-api:1.0.0')
}

kotlin.experimental.coroutines = 'enable'

compileKotlin {
    kotlinOptions.jvmTarget = "1.8"
}
compileTestKotlin {
    kotlinOptions.jvmTarget = "1.8"
}

task run(dependsOn: appengineRun)

appengine {
    deploy {
    version = 'media-staging-1201181257pm'
    }
}

src/main/resources/application.conf

ktor {
    application {
        modules = [ InitializationKt.main ]
    }
}

src/main/webapp/WEB-INF/

appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <threadsafe>true</threadsafe>
    <runtime>java8</runtime>
</appengine-web-app>

web.xml

<?xml version="1.0" encoding="ISO-8859-1" ?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0">
    <servlet>
        <display-name>KtorServlet</display-name>
        <servlet-name>KtorServlet</servlet-name>
    <servlet-class>io.ktor.server.servlet.ServletApplicationEngine</servlet-class>
        <!-- path to application.conf file, required -->
        <init-param>
            <param-name>io.ktor.config</param-name>
            <param-value>application.conf</param-value>
        </init-param>
    </servlet>
    <servlet-mapping>
        <servlet-name>KtorServlet</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>
</web-app>

Thanks for the help!

Server

Json request failure with configured form authentication

https://youtrack.jetbrains.com/issue/KTOR-678

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/926

Ktor Version

1.1.2

Ktor Engine Used(client or server and name)

Netty server

JVM Version, Operating System and Relevant Context

Java 1.8, Linux

Feedback

I have the following configuration:

fun Application.main() {
    install(ContentNegotiation) {
        jackson {
        }
    }
    install(Authentication) {
        form {
            challenge = FormAuthChallenge.Unauthorized
            validate { credentials ->
                if (credentials.name == credentials.password) {
                    UserIdPrincipal(credentials.name)
                } else {
                    null
                }
            }
        }
    }
    routing {
        authenticate {
            post("/test") {
                call.respondText("OK")
            }
        }
    }
}

I should get "401 Unauthorized" response when making a simple POST request on "/test" endpoint. But when the content type of the request is "application/json", generated for instance with the following curl command:

curl -v -d '{}' -H "Content-Type: application/json" -X POST http://localhost:8080/test

ktor server fails with "500 Internal Server Error" and the exception:

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of
`io.ktor.http.Parameters` (no Creators, like default construct, exist): abstract types either need to be
mapped to concrete types, have custom deserializer, or contain additional type information
 at [Source: (InputStreamReader); line: 1, column: 1]
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1452)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1028)
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserialize(AbstractDeserializer.java:265)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4013)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3049)
        at io.ktor.jackson.JacksonConverter.convertForReceive(JacksonConverter.kt:40)
        at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:104)
        at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
        at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:69)
        at io.ktor.request.ApplicationReceiveFunctionsKt.receiveOrNull(ApplicationReceiveFunctions.kt:85)
        at io.ktor.auth.FormAuthKt$form$1.invokeSuspend(FormAuth.kt:93)
        at io.ktor.auth.FormAuthKt$form$1.invoke(FormAuth.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
        at io.ktor.auth.Authentication.processAuthentication(Authentication.kt:217)
        at io.ktor.auth.Authentication$interceptPipeline$2.invokeSuspend(Authentication.kt:112)
        at io.ktor.auth.Authentication$interceptPipeline$2.invoke(Authentication.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
        at io.ktor.routing.Routing.executeResult(Routing.kt:148)
        at io.ktor.routing.Routing.interceptor(Routing.kt:29)
        at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:93)
        at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:63)
        at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:106)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
        at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:31)
        at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
        at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
        at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:160)
        at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
        at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
        at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
        at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

HTTP/2 push fails with Netty engine

https://youtrack.jetbrains.com/issue/KTOR-800

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/350

The problem reproduces with http2-push from https://github.com/ktorio/ktor-samples

Run ./gradlew :http2-push:run
Navigate to https://localhost:8443/

Resulting exception:

2018-03-07 14:47:07.089 [nettyWorkerPool-3-2] WARN  i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
2018-03-07 14:47:07.093 [nettyWorkerPool-3-2] WARN  i.n.util.concurrent.DefaultPromise - An exception was thrown by io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete()
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
        at io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete(NettyHttp2Handler.kt:109)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
        at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163)
        at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:108)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        ... 7 common frames omitted

Clearing Session Cookie in Chrome 80+ with SameSite and Secure

When unable to get JWKS, JWTAuth swallows the underlying exception and only logs the last message

https://youtrack.jetbrains.com/issue/KTOR-636

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1053

https://github.com/ktorio/ktor/blob/2406dc5b5970205593efa51867c0e5a05f149210/ktor-features/ktor-auth-jwt/jvm/src/io/ktor/auth/jwt/JWTAuth.kt#L164

When unable to retrieve the JWKS, JWTAuth swallows the underlying exception and only logs the message.

This means we only get to see:
TRACE io.ktor.auth.jwt - Failed to get JWK: Failed to get key with kid 1

When stack trace is:

com.auth0.jwk.SigningKeyNotFoundException: Failed to get key with kid 1
Caused by: com.auth0.jwk.SigningKeyNotFoundException: Cannot obtain jwks from url https://localhost:2222/jwks
Caused by: java.net.ConnectException: Connection refused (Connection refused)

Simple example to reproduce:

val TestJWT = "eyJraWQiOiIxIiwiYWxnIjoiUlMyNTYifQ.eyJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwNjAvaXNzby9vYXV0aDIiLCJleHAiOjE3MTE4NzQ3MDAsImp0aSI6IjZvWkZWa3lGX2RSd1dpcXNhQkJkWkEiLCJpYXQiOjE1NTQxOTQ3MDAsInN1YiI6Im15dXNlciIsImF1ZCI6Ik9JREMiLCJhY3IiOiJMZXZlbDQiLCJhenAiOiJPSURDIn0.NuWfe1BZK3i-VVC1l7EIvJydd9m3Pcr2_0AanhbS3YEXSq_NWKhqtFd4qM_KUhLURTTwhNhAb43Zr2HzxGFUhnYnU4uCi95fLcw3Cq8mTM3o4I0r-pgpPTkfiheUUtOA4d43cwWpyEaBdypwO_F-VLA4zBw1oTRE_M0_G-16Q6yezpjTVBvOI7nsEWLHUZ-i10hE3V53cx2-Qm5OUOtEFF-UqqFhgBU6VSRYS5J3puWQFGlLr5hGSAW3Nll1DkJbiNaHB4y7EPnSlCPcNdZ98PXckylsiJ6nhRJXg4mke-C2WWckJ5H4dgsjeoUmXDuLekO1IrvwT1JLGJYiPwlQJw"

fun main(args: Array<String>): Unit {
    Thread {
        io.ktor.server.netty.EngineMain.main(args)
    }.run()

    Thread {
        println("Testing in 3 secs...")
        Thread.sleep(3000);
        println("Testing...")
        GlobalScope.launch {
            val resp = HttpClient(Apache).use { client ->
                client.get<String>(URL("http://localhost:8080/")) {
                    this.header("Authorization", "Bearer " + TestJWT);
                }
            }
            println("RESP: ${resp}")
        }
    }.run()
}

@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(Authentication) {
        jwt {
            verifier(JwkProviderBuilder(URL("https://localhost:2222/jwks")).build(), "http://localhost:8060/isso/oauth2");
            validate { credentials ->
                if (credentials.payload.subject == "myuser") {
                    JWTPrincipal(credentials.payload)
                } else {
                    log.info("${credentials.payload.subject} is not authorized to use this app, denying access")
                    null
                }
            }
        }
    }
    routing {
        authenticate {
            get("/") {
                call.respondText("HELLO WORLD!", contentType = ContentType.Text.Plain)
            }
        }
    }
}

How to get remote (client) port

Using existing event-loop group

support application.conf with env type

Increase max HTTP header size for server

Exception or warning on mismatch with optional path param and non default valued parameter of location class

Support for unsigned types

Support setting caching options on call

https://youtrack.jetbrains.com/issue/KTOR-457

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1688

Subsystem
CachingHeaders feature 1.3.1 and its docs

Is your feature request related to a problem?
I was trying to set up a cache for an endpoint, but the request has to be conditionally cached depending on whether the user is logged in. i.e.

get<Cinemas.Routes.List> {
	if (call.hasUser) {
		// this cannot be cached, because the user can change cinema's state (favorite it)
		call.respond(repository.getCinemasAuth(userID = call.userId))
	} else {
		// no logged in user, safe to cache for days/weeks since the cinema list barely ever changes
		call.cache(CachingOptions(/* parameters specifically for this response path */))
		call.respond(repository.getActiveCinemas())
	}
}

Describe the solution you'd like
The implementation of ApplicationCall.cache() was really tricky. The documentation only shows how to cache centrally based on content type. No mention of caching by route or code path. Reading the code and debugging for hours I found that the following setup works:

embeddedServer(Netty, port) { // this: Application
	install(CachingHeaders) {
		// default is `options { it.caching }`, see ktorio/ktor#1687
	}
	routing {
		get ... // as above
	}
}
fun ApplicationCall.cache(caching: CachingOption) {
	this.response.pipeline.intercept(ApplicationSendPipeline.Render) {
		// by definition of Render phase, it can be only OutgoingContent here
		// this will be read by the default CachingHeaders.options lambda
		// and put it among the response headers
		(subject as OutgoingContent).caching = caching
	}
}

Motivation to include to ktor
A function like call.cache(...) above or a property call.caching = ... would be beneficial to share via in-built extensions, so that users who write Ktor apps can set caching up easily.

It is possible that what's missing is a lambda-bearing overload of respond(Any) similar to respondText, where OutgoingContent can be amended.

CORS: Pattern matching for origin

Client does not support sending or receiving json null value

Incoming request body validation

Support for adding values to the MDC later on in the pipeline.

Netty: no way of configuring client certificate auth

Locatons: cannot use instance standalone

Websockets and Netty weird exception

How to monitor whether ktor is running and restart server automatically if there's an exception?

Problem with websockets when deploying to a jetty server as a war

Change default log level for CallLogging feature to INFO

When returning a String, content negotiation is ignored

Functionality to resolve the user's email or ID from a token for popular providers

Isolated module for connector

Locations: Support more than one path

Provide /** alias for tailcard /{...}

Is there a blueprint name like the python flask?

What's the recommended way to implement advanced use cases such as API rate limiting or response caching?

Ability to set version headers in the application call

Regression in routing / locations routing

Data conversion with generic

Support 'group' construction for grouping routes

How do I find out the request handler before processing?

https://youtrack.jetbrains.com/issue/KTOR-764

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/503

How do I find out the request handler before processing?

I made a custom feature that should check the user's permissions to use the request. Can I monitor what handler the router is accessing?
Can this looks like that?
if (!User.accessTo.contains(REQUEST_HANDLER_NAME_OR_SOMETHING)){
call.respond(HttpStatusCode.BadRequest) }

import io.ktor.application.*
import io.ktor.http.HttpStatusCode
import io.ktor.pipeline.*
import io.ktor.response.respond
import io.ktor.sessions.*
import io.ktor.util.AttributeKey
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

data class UserRights(
        val haveFullAccess:Boolean,
        val accessTo:List<String>,
        val canUpdate:Boolean,
        val canDelete:Boolean,
        val canBan:Boolean,
        val canMute:Boolean)

var User = UserRights(false, listOf(""),false,false,false,false)

class RightsChecker(configuration: Configuration) {
    val prop = configuration.prop // get snapshot of config into immutable property
    class Configuration {
        var prop = "value"
    }
    companion object Feature : ApplicationFeature<ApplicationCallPipeline, Configuration, RightsChecker> {
        override val key = AttributeKey<RightsChecker>("RightsChecker")
        override fun install(pipeline: ApplicationCallPipeline, configure: Configuration.() -> Unit): RightsChecker {
            val configuration = RightsChecker.Configuration().apply(configure)
            val feature = RightsChecker(configuration)

            val FilterPhase = PipelinePhase("CallFilter")
            pipeline.insertPhaseAfter(ApplicationCallPipeline.Infrastructure, FilterPhase)

            pipeline.intercept(FilterPhase) {
                val session = call.sessions.get<SessionData>() ?: SessionData(0, "Guest")
                println(LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_TIME) + " User ${session.userID}, ${session.role} connected")
                when (session.role) {
                    "Guest" -> User = UserRights(
                            haveFullAccess = false,
                            accessTo = listOf(""),
                            canUpdate = false,
                            canDelete = false,
                            canBan = false,
                            canMute = false)
                    "User" -> User = UserRights(
                            haveFullAccess = false,
                            accessTo = listOf("lUsers"),
                            canUpdate = false,
                            canDelete = false,
                            canBan = false,
                            canMute = false)
                    "Moder" -> User = UserRights(
                            haveFullAccess = false,
                            accessTo = listOf("lUsers"),
                            canUpdate = true,
                            canDelete = false,
                            canBan = false,
                            canMute = true)
                    "Admin" -> User = UserRights(
                            haveFullAccess = true,
                            accessTo = listOf("lUsers"),
                            canUpdate = true,
                            canDelete = true,
                            canBan = true,
                            canMute = true)
                }
                if (!User.accessTo.contains(**REQUEST_HANDLER_NAME_OR_SOMETHING**)){
                    call.respond(HttpStatusCode.BadRequest)
                }
            }

            pipeline.intercept(ApplicationCallPipeline.Call) {
                val session = call.sessions.get<SessionData>() ?: SessionData(0,"Guest")
                println(LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_TIME)+" User ${session.userID}, ${session.role} call answered")
            }
            return feature
        }
    }
}

Location parameters with invalid kotlin names

Is it possible to have multiple ContentNegotiation, separate per module?

embeddedServer for CIO and Netty inconsistency

Locations: Support trailing /

Serialization for sealed classes gives different results when used through Ktor

Creating HttpClient with Apache engineFactory doesn't work

https://youtrack.jetbrains.com/issue/KTOR-589

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1234

Ktor Version and Engine Used (client or server and name)
version, client/server, additional ktor modules included
ktor 1.2.2
kotlin 1.3.41

dependencies

compile "ch.qos.logback:logback-classic:$logback_version"
    compile "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.9.2"
    compile "com.google.appengine:appengine:$appengine_version"
    compile "com.google.cloud:google-cloud-logging-logback:$gce_logback_version"
    compile "io.ktor:ktor-auth:$ktor_version"
    compile "io.ktor:ktor-client-apache:$ktor_version"
    compile "io.ktor:ktor-client-auth-jvm:$ktor_version"
    compile "io.ktor:ktor-client-core-jvm:$ktor_version"
    compile "io.ktor:ktor-client-core:$ktor_version"
    compile "io.ktor:ktor-html-builder:$ktor_version"
    compile "io.ktor:ktor-jackson:$ktor_version"
    compile "io.ktor:ktor-locations-metadata:$ktor_version"
    compile "io.ktor:ktor-locations:$ktor_version"
    compile "io.ktor:ktor-server-core:$ktor_version"
    compile "io.ktor:ktor-server-netty:$ktor_version"
    compile "io.ktor:ktor-server-servlet:$ktor_version"
    compile "net.logstash.logback:logstash-logback-encoder:5.1"
    compile "org.jetbrains:kotlin-css-jvm:1.0.0-pre.31-kotlin-1.2.41"
    compile 'com.google.cloud:google-cloud-firestore:1.11.0'
    compile 'com.google.cloud:google-cloud-storage:1.38.0'
    compile 'org.koin:koin-ktor:2.0.1'
    testCompile "io.ktor:ktor-server-tests:$ktor_version"
    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"

Intelij:

IntelliJ IDEA 2019.1.3 (Ultimate Edition)
Build #IU-191.7479.19, built on May 28, 2019
JRE: 1.8.0_202-release-1483-b58 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.1

Describe the bug
When installin Authentication module I need to create an Apache client. Doing that by using the constructor with engineFactory causes a weird error where Inteliji syntax is almost error like colour (#C53A47). When hovering over - no explanation, when I start the app it says that client lateinit var has never been initialized.

To Reproduce
Follow official tutorial for adding authentication.

Expected behavior
Adding http client by following factory object (Apache).

My workaround (1st line on the screenshot) is to use the constructor with engine (not engineFactory) and calling build() on Apache object.

Screenshots
Screenshot 2019-07-17 at 20 54 53

Please accept my apologies if I posted this in a wrong place or this is obvious. It's just my best intention to help others to avoid this behaviour.

Thanks

@Location annotation for typealias target

Performance Issue / Ktor & Netty

httpMethod is not affected by X-Http-Method-Override (in opposite to docs)

PartialContent need responed content length when respcode = 206

Feature: Use websockets with serialization

corsCheckRequestHeaders false

Session cookie with BASE64 encoding fails to set correct cookie

Content Negotiation: Gson: Should be able to return 400 for badly formatted request.

Build in feature for Single PAge Applications

Enhance api for ConditionalHeaders usage

LinkageError: loader constraint violation

https://youtrack.jetbrains.com/issue/KTOR-434

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1744

I think this issue is related to #783

Ktor Version
1.3.1

Ktor Engine Used
server netty

JVM Version, Operating System, and Relevant Context
jdk1.8.0_201 ，Linux Mint 19

I was using gradle application plugin

running
gradle run
while on dev it was running fine

but for the case using the distribution created after
gradle build

project_dir/build/distribution when I it tried runing the application

./application_name

initial loading like creating database and redis connection instance is successful then after that the error occurred

I encountered the same issue #783

These are my dependencies:

plugins {
    java
    kotlin("jvm")
    application
}


val ktor_version = "1.3.1"
val kodein_verison = "6.5.3"
val exposedVersion = "0.22.1"
val csv_reader_version = "0.7.3"

repositories {
    mavenCentral()
    jcenter()
    maven("https://jitpack.io")
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))

    //-- local jar files
    implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))

    //ktor
    implementation("io.ktor:ktor-server-netty:$ktor_version")
    implementation("io.ktor:ktor:$ktor_version")
    implementation("io.ktor:ktor-websockets:$ktor_version")
    implementation( "io.ktor:ktor-gson:$ktor_version")

    //logging
    implementation("io.github.microutils:kotlin-logging:1.6.24")
    implementation("ch.qos.logback:logback-classic:1.2.3")

    // token verification
    implementation("io.ktor:ktor-auth-jwt:$ktor_version")

    //json handlers
    implementation("com.beust:klaxon:5.0.1")

    //encryption
    implementation("com.github.simbiose:Encryption:2.0.1")
    implementation("commons-codec:commons-codec:1.14")

    //di
    implementation("org.kodein.di","kodein-di-generic-jvm",kodein_verison)

    //redis
    implementation("io.lettuce:lettuce-core:5.2.1.RELEASE")

    //database
    implementation("org.jetbrains.exposed", "exposed-core", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-dao", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-jdbc", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-java-time", exposedVersion)

    //csv reader
    implementation("com.github.doyaaaaaken:kotlin-csv-jvm:$csv_reader_version")

    implementation("com.zaxxer:HikariCP:3.4.1")
    implementation ("mysql:mysql-connector-java:8.0.11")

    //-- jasper reports
    implementation("net.sf.jasperreports", "jasperreports", "6.4.0") {
        exclude("org.olap4j", "olap4j")
    }
    implementation("com.lowagie", "itext", "2.1.7")
}

application conf

ktor {
    enviroment = dev
    deployment {
        port = 3001
        watch = ["project_root"]
    }

    application {
        modules = [
            com.my.module1,
            com.my.module2,
        ]
    }
}

stack trace:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at kotlin.reflect.jvm.internal.calls.CallerImpl$Method.callMethod(CallerImpl.kt:97)
        at kotlin.reflect.jvm.internal.calls.CallerImpl$Method$Static.call(CallerImpl.kt:106)
        at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:106)
        at kotlin.reflect.jvm.internal.KCallableImpl.callDefaultMethod$kotlin_reflection(KCallableImpl.kt:152)
        at kotlin.reflect.jvm.internal.KCallableImpl.callBy(KCallableImpl.kt:110)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.callFunctionWithInjection(ApplicationEngineEnvironmentReloading.kt:380)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:328)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1$$special$$inlined$forEach$lambda$1.invoke(ApplicationEngineEnvironmentReloading.kt:275)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1$$special$$inlined$forEach$lambda$1.invoke(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:308)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:274)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:290)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:272)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:125)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:245)
        at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:126)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:26)
        at com.vhl.sms.caster.MainAppKt.main(MainApp.kt:47)
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method "io.ktor.application.ApplicationEvents.subscribe(Lio/ktor/application/EventDefinition;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle;" the class loader (instance of io/ktor/server/engine/OverridingClassLoader$ChildURLClassLoader) of the current class, io/ktor/websocket/WebSockets$Feature, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for the method's defining class, io/ktor/application/ApplicationEvents, have different Class objects for the type kotlinx/coroutines/DisposableHandle used in the signature
        at io.ktor.websocket.WebSockets$Feature.install(WebSockets.kt:112)
        at io.ktor.websocket.WebSockets$Feature.install(WebSockets.kt:104)
        at io.ktor.application.ApplicationFeatureKt.install(ApplicationFeature.kt:64)
        at com.vhl.sms.caster.MainAppKt$messageModule$1.invokeSuspend(MainApp.kt:105)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:271)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.vhl.sms.caster.MainAppKt.messageModule(MainApp.kt:57)
        ... 25 more

Setting Content-Length Header manually when using call.respondOutputStream

Cannot log status code with MDC

JSON RPC support

Make default charset UTF-8 when using `receiveText` for application/json request

StatusPages: Testing of application fails because of null status code

XForwardedHeaderSupport should let you specify which index (from end) to choose

[Evolution] Ktor locations nested classes

https://youtrack.jetbrains.com/issue/KTOR-467

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1660

Subsystem
ktor-locations

Description
We are going to develop and expand ktor Locations (Type-Safe routing https://ktor.io/servers/features/locations.html). One of the directions to improve is Kotlin Multiplatform. The most obvious way of migration to MPP is to use kotlinx.serialization in locations. Due to the limited reflection capabilities of non-JVM targets, there are things that not so easy to implement. On the other hand, there are questionable features that may lead to issues. One of the problematic features is nested location classes and nested location objects.

What we are thinking of to change:

a nested location class should always have a property of the outer class or object
nested objects in objects are not allowed

The motivation for the first point is the fact that a location class nested to another, makes no sense without the ability to refer to the outer class. Consider the following example:

@Location("/api/{version}")
class Api(val version: Int) {
    @Location("/user/{id}")
    class User(val id: String, val api: Api = Api(1)) {
        // here API presence looks required, 
        // otherwise, there is no way to find a version value
        // so now we demand the api parameter to exist
    }
}

The other point is that one can't just move a nested class outside without additional manual changes:

@Location("/root")
class Outer {
    @Location("/child")
    class Child  // it is tied to /root/child path
}

@Location("/child")
class Child  // this is tied to just /child

@Location("/child2")
class Child2(val outer: Outer) // -> /root/child2

The cause of the second change is that objects should be symmetric with classes, and one can't add a constructor property to an object. The unfortunate consequence is that one can't write like this anymore:

@Location("/api")
object Api {
    @Location("/info")
    object Info 
}

Should be migrated to the following:

@Location("/api")
object Api {
    @Location("/info")
    class Info(val api: Api = Api)
    // the api parameter is required, weird isn't it
}

Server for Kotlin Native

java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled

https://youtrack.jetbrains.com/issue/KTOR-472

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1652

Hi all
When I run my Ktor application with gradle run then I've got the following exception:

19:21:11.795 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 11
19:21:11.811 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled
        at io.netty.util.internal.ReflectionUtil.trySetAccessible(ReflectionUtil.java:31)
        at io.netty.util.internal.PlatformDependent0$4.run(PlatformDependent0.java:225)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:219)
        at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
        at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
        at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
19:21:11.815 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
java.lang.IllegalAccessException: class io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @557caf28
        at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
        at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
        at java.base/java.lang.reflect.Method.invoke(Method.java:558)
        at io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:326)
        at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
        at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
        at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)

the content of build.gradle.kts file

plugins {
    application
    kotlin("jvm") version "1.3.61"
}

group = "io.flatmap"
version = "1.0-SNAPSHOT"


val ktor_version = "1.3.0"

repositories {
    mavenCentral()
    jcenter()
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))
    compile("io.ktor:ktor-server-netty:$ktor_version")
    compile("io.ktor:ktor-server-core:$ktor_version")
    compile("ch.qos.logback:logback-classic:1.2.3")
    testCompile(group = "junit", name = "junit", version = "4.12")
}

tasks {
    compileKotlin {
        kotlinOptions.jvmTarget = "11"
    }
    compileTestKotlin {
        kotlinOptions.jvmTarget = "11"
    }
}

application {
    mainClassName = "io.ktor.server.netty.EngineMain"
}

I am using Zulu OpenJDK 11:

 java --version
openjdk 11.0.6 2020-01-14 LTS
OpenJDK Runtime Environment Zulu11.37+17-CA (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.37+17-CA (build 11.0.6+10-LTS, mixed mode)

I thought, this is solved through https://github.com/ktorio/ktor/issues/1190.

Thanks

XForwardedHeaderSupport is installed late in the pipeline

respondOutputStream behind nginx fails

https://youtrack.jetbrains.com/issue/KTOR-346

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1933

Versions: JVM OpenJDK 11
OS: Dockerized Ubuntu ARM64, Raspberry Pi 4
Ktor: 1.3.2
Kotlin: 1.3.72

Code here, docker-compose here, nginx conf here.

when using call.respond() it works fine. Why call.respondOutputStream() fails?

EDIT: After digging in Nginx debug logs it appears that:

So the upstream server returns a response with "Transfer-Encoding: chunked", and the first byte of the response body is 0x89. This character is outside of the ASCII range and clearly is not a hexadecimal number as required by the chunked encoding.
Clearly this is an incorrect response from the backend. Depending on how the framework you are using is expected to work, this may either indicate a bug in the framework, or you are using the framework incorrectly.

Cross reference ticket from Nginx issue board.

Nginx error:

upstream sent invalid chunked response while reading upstream

Error stack:

ERROR ktor.application - 200 OK: (request error: java.lang.IllegalStateException: servletRequest.method must not be null)
java.lang.IndexOutOfBoundsException: null
	at javax.imageio.stream.FileCacheImageOutputStream.seek(FileCacheImageOutputStream.java:170)
	at javax.imageio.stream.FileCacheImageOutputStream.close(FileCacheImageOutputStream.java:231)
	at javax.imageio.ImageIO.write(ImageIO.java:1590)
...
Suppressed: java.io.IOException: io.ktor.util.cio.ChannelWriteException: Failed to write to servlet async stream
		at io.ktor.utils.io.jvm.javaio.OutputAdapter.close(Blocking.kt:122)
		at kotlin.io.CloseableKt.closeFinally(Closeable.kt:56)
		... 8 common frames omitted
	Caused by: io.ktor.util.cio.ChannelWriteException: Failed to write to servlet async stream
		at io.ktor.server.servlet.ServletWriter.wrapException(ServletWriter.kt:119)
		at io.ktor.server.servlet.ServletWriter.onError(ServletWriter.kt:112)
		at io.ktor.server.servlet.ServletWriter.run(ServletWriter.kt:52)
		at io.ktor.server.servlet.ServletWriter$run$1.invokeSuspend(ServletWriter.kt)
		at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
...
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
		at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:309)
		at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:272)
		at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:118)
		at io.ktor.server.servlet.ServletWriter.loop(ServletWriter.kt:87)
		at io.ktor.server.servlet.ServletWriter$loop$1.invokeSuspend(ServletWriter.kt)
		... 46 common frames omitted
	Caused by: java.io.IOException: Broken pipe
		at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
		at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
		at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
		at sun.nio.ch.IOUtil.write(IOUtil.java:65)
		at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:468)
		at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:138)
...

Receive JSON formatted POST data into ValuesMap with Gson

Jackson: receiveOrNull crashes with an exception when sending empty content

Jackson-backed `ApplicationCall.receive` does not throw `ContentTransformationException`

Specify source of parameter for Ktor Location

Manage feature dependencies

insertPhaseBefore and insertPhaseAfter lead to different order

Noticeable lag while starting and stopping Ktor

ktor with with mustache 415 Unsupported Media Type: POST

Intermittent 500 errors with empty bodies with Jetty

Unhandled get

https://youtrack.jetbrains.com/issue/KTOR-573

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1279

A few times a second my screen is filled with:

2019-08-12 06:17:03.171 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:06.197 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:09.232 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:12.266 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:13.205 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:15.301 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:18.336 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:21.369 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:23.238 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:24.404 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:24.761 [nioEventLoopGroup-4-1] INFO  Application - 302 Found: GET - / -> /static/index.html
2019-08-12 06:17:24.810 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/index.html
2019-08-12 06:17:24.878 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/css/reset.css
2019-08-12 06:17:24.900 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/css/mainstyle.css
2019-08-12 06:17:24.927 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/nav/nav_over_01.png
2019-08-12 06:17:24.935 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/nav/nav_05.png
2019-08-12 06:17:25.325 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/diracian.png
2019-08-12 06:17:25.340 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/alto-whitelabel.png
2019-08-12 06:17:25.356 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/andaz-whitelabel.jpg
2019-08-12 06:17:25.369 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/payscout-whitelabel.png

The only difference from a request that works is the GET is lowercase and the other is uppercase. I don't see any mention in RFC 7231 that says that the verb needs to be capitalized. However, they do make a point to capitalize it through the entire document. Is there a way for me to handle these unhandled requests?

ldap any credentials get authenticated

Multiplatform: Ktor can't find file from distributions folder in a child module

When i check opened server port i got Exception (CIO)

https://youtrack.jetbrains.com/issue/KTOR-517

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1469

ktor 1.2.6
server is io.ktor:ktor-server-cio

I have exception if try check open server port over nmap

Steps:

Start server
Scan server use nmap... "nmap -P0 DEVICE_IP -p 8999"

2019-12-02 17:55:34.917 9591-9669/? E/AndroidRuntime: FATAL EXCEPTION: DefaultDispatcher-worker-3
Process: org.jajaz.wearreader, PID: 9591
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:419)
at kotlinx.io.nio.ChannelsKt.read(Channels.kt:117)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:75)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invoke(Unknown Source:10)
at kotlinx.coroutines.io.ByteBufferChannel.writeSuspendSession(ByteBufferChannel.kt:1931)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1.invokeSuspend(CIOReader.kt:65)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1.invoke(Unknown Source:10)
at kotlinx.coroutines.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:123)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedKt.resumeCancellable(Dispatched.kt:457)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch(Unknown Source:1)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:47)
at kotlinx.coroutines.BuildersKt.launch$default(Unknown Source:1)
at kotlinx.coroutines.io.CoroutinesKt.launchChannel(Coroutines.kt:118)
at kotlinx.coroutines.io.CoroutinesKt.writer(Coroutines.kt:76)
at io.ktor.network.sockets.CIOReaderKt.attachForReadingDirectImpl(CIOReader.kt:61)
at io.ktor.network.sockets.NIOSocketImpl$attachForReading$1.invoke(NIOSocket.kt:45)
at io.ktor.network.sockets.NIOSocketImpl$attachForReading$1.invoke(NIOSocket.kt:18)
at io.ktor.network.sockets.NIOSocketImpl.attachFor(NIOSocket.kt:80)
at io.ktor.network.sockets.NIOSocketImpl.attachForReading(NIOSocket.kt:41)
at io.ktor.network.sockets.SocketsKt.openReadChannel(Sockets.kt:111)
at io.ktor.server.cio.HttpServerKt$httpServer$acceptJob$1.invokeSuspend(HttpServer.kt:109)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)

Sudden clusters of Channel failed logs

RequestQueueLimit and other Netty configuration strange behavior

https://youtrack.jetbrains.com/issue/KTOR-568

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1298

Ktor Version and Engine Used (client or server and name)
Ktor 1.2.3 + Netty

Describe the bug
Hello, maybe this is not a bug, and I just don’t understand how this should work, but in my opinion there may be a mistake.

I create a server with this code (imagine that debugPlay does some work for +-200 ms):

fun main() {
val env = applicationEngineEnvironment {
        // Public API
        connector {
            port = 1717
        }
        module {
            mainGet()
        }
    }
    embeddedServer(Netty, environment = env, configure = {
        connectionGroupSize = 1
        workerGroupSize = 1
        callGroupSize = 1
        runningLimit = 1
        requestQueueLimit = 1
    }).start(true)
}
}

fun Application.mainGet() {
    install(CallLogging)
    routing {
        get("/api/checkThis") {
            call.respond(TextContent("""{"msg": "unaccessable when server process old queue request"}""", ContentType.Application.Json))
        }
        get("/api/debugPlay") {
            Thread.sleep(200)
            println("debugPlay")
            call.respond(TextContent(JSONObject().put("sign", "123").toString(), ContentType.Application.Json))
            return@get
       }
    }
}

I set all available configure options to 1 to demonstrate the problem.
So, I expect that the queue can save no more than one request. Next, I send 1000 requests from another application simultaneously and close it without waiting for an answer.

However, Ktor continues to respond to requests for a long time, despite the fact that only one request should remain in the queue. Here is the log:

debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
...

If I do not install Netty options and just use the standard configuration, then the server stops responding to other requests (to another route), and continues to process old ones that have already been canceled. Thus, it is very easy to make a ddos attack by sending a lot of requests to which the server will respond, while ignoring new requests from real users. Can you help: is saving old requests in the queue an error (since I set a limit of 1 request), or I just don’t understand why it is so right? And how to optimally configure ktor so as not to become an easy target for a ddos attack?)

Locations: Type class XYZ is not supported in default data conversion service

https://youtrack.jetbrains.com/issue/KTOR-569

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1296

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-locations:1.2.3")
implementation("io.ktor:ktor-serialization:1.2.3")

Describe the bug
Default data conversions fails when nesting Object -> Class -> Object -> Class with the @Location annotation like in the following example (Branch is an object, List a class):


val branchList = Organization.ById.Branch.List(Organization.ById("abc"))
// Would become "/organizations/abc/branches"
val branchListHref = application.locations.href(organizationBranchList)

It yields the following error:

Type class Organization$ById is not supported in default data conversion service
io.ktor.util.DataConversionException: Type class Organization$ById is not supported in default data conversion service
	at io.ktor.util.DefaultConversionService.toValues(ConversionService.kt:46)
	at io.ktor.locations.Locations.pathAndQuery(Locations.kt:210)
	at io.ktor.locations.Locations.href(Locations.kt:222)
	at ...ApplicationTest$branch offices for organization$1.invoke(ApplicationTest.kt:183)

A similar version using nested Object -> Class -> Class works like expected (BranchList is a class):

val branchList = Organization.ById.BranchList(Organization.ById("abc"))
val branchListHref = application.locations.href(organizationBranchList)

Both are based on the following declaration of API routes:

@Location("/organizations")
object Organization {
    @Location("/")
    class List(val offset: Int = 0, val limit: Int = 20)
    @Location("/{id}")
    class ById(val id: String) {
        // 1)  Does not work
        @Location("/branches")
        object Branch {
            @Location("/")
            class List(val organization: Organization.ById, val offset: Int = 0, val limit: Int = 20)
            @Location("/{id}")
            class ById(val organization: Organization.ById, val id: String)
        }

        // 2) Works 
        @Location("/branches/")
        class BranchList(val organization: ById, val offset: Int = 0, val limit: Int = 20)
        @Location("/branches/{id}")
        class BranchById(val organization: ById, val id: String)
    }
}

Style 2) seems more idiomatic as it allows to POST or PUT to Organization.ById.Branches (/organizations/123/branches) to create a new Branch.

To Reproduce
See above code example.

Expected behavior
Arbitrary nesting of locations should work.

Second authentication method is ignored

https://youtrack.jetbrains.com/issue/KTOR-606

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1191

Ktor Version

1.2.1

Ktor Engine Used (client or server and name)

Netty

JVM Version, Operating System and Relevant Context

JDK 8, Mac

Issue

I'm building a multi-layer custom authentication for my web service. I declare it like this:

// in Application.module()
install(Authentication) {
  appId(AUTH_APP_ID) { // this one is my custom one
    validate {
      if (AuthorizedApps.isReadAuthorized(it.appId)) // simplified logic
        AppIdPrincipal(AuthorizedApps.getAppName(it.appId)!!)
      else null
    }
  }
}

And routes look like this:

routing {
  authenticate(AUTH_APP_ID) {
    get("/info", controller<ServiceInformationController>()) // controller handles everything
  }
}

So... this works as expected. It was actually much easier than anticipated.

THE PROBLEM

Now I want to add another custom authentication, for example "AppSecret". Same as ID, just handles a different header (app_secret) along with this app_id header.

I won't paste the whole thing here (see the bottom for more info), but it's pretty much the same.. except for this part:

authenticate(AUTH_APP_ID, AUTH_APP_SECRET) { // observe: two authentication methods
  get("/info", controller<ServiceInformationController>())
}

I also tried the other approach, with nested authenticate blocks.

In both cases, the second authentication method is ignored.

The truth is.. yes, right now, I only need the one (secret), but let's say in the future I want to add another one, that I want to combine with the others.. for example JWT or something on top of AppID/AppSecret combo.

Why is my second authentication method completely ignored?
I debugged this, Ktor never even tries after the first auth method is successful.

Here's my custom authentication for reference (AppID):

package tokenizer.v1.security

import io.ktor.application.ApplicationCall
import io.ktor.application.call
import io.ktor.auth.Authentication
import io.ktor.auth.AuthenticationFunction
import io.ktor.auth.AuthenticationPipeline
import io.ktor.auth.AuthenticationProvider
import io.ktor.auth.Credential
import io.ktor.auth.Principal
import io.ktor.http.HttpStatusCode
import io.ktor.request.ApplicationRequest
import io.ktor.request.header
import io.ktor.response.respond
import tokenizer.v1.Configuration.Headers
import tokenizer.v1.security.AppIdAuthenticationProvider.Configuration

data class AppIdCredential(val appId: String) : Credential

data class AppIdPrincipal(val appName: String) : Principal

class AppIdAuthenticationProvider(configuration: Configuration) : AuthenticationProvider(configuration) {
  class Configuration(name: String?) : AuthenticationProvider.Configuration(name) {
    // provides a validation function that will check given `AppIdCredential` instance
    // and return `Principal` (or `null` if credential does not correspond to an authenticated principal)
    var authenticationFunction: AuthenticationFunction<AppIdCredential> = { null }

    fun validate(body: suspend ApplicationCall.(AppIdCredential) -> Principal?) {
      authenticationFunction = body
    }
  }

  val authenticationFunction = configuration.authenticationFunction
}

// installs an app ID authentication mechanism
fun Authentication.Configuration.appId(name: String? = null, configure: Configuration.() -> Unit) {
  val provider = AppIdAuthenticationProvider(Configuration(name).apply(configure))
  val authenticateFunction = provider.authenticationFunction

  provider.pipeline.intercept(AuthenticationPipeline.RequestAuthentication) { context ->
    val credential = call.request.appIdAuthenticationCredential()
    val principal = credential?.let { authenticateFunction(call, it) }

    val cause = when {
      credential == null -> call.respond(HttpStatusCode.Forbidden)
      principal == null -> call.respond(HttpStatusCode.Unauthorized)
      else -> null
    }

    if (cause != null) {
      // maybe respond with something? probably not...
    }

    principal?.let { context.principal(it) }
  }

  register(provider)
}

// retrieves App ID Credential for this `ApplicationRequest`
@Suppress("MoveVariableDeclarationIntoWhen")
fun ApplicationRequest.appIdAuthenticationCredential(): AppIdCredential? {
  val appId = header(Headers.APP_ID)
  // we need a valid app ID
  if (appId.isNullOrBlank()) return null
  // we have "something", so let's use this
  return AppIdCredential(appId)
}

Error serving static video: Failed to load resource: net::ERR_INCOMPLETE_CHUNKED_ENCODING

https://youtrack.jetbrains.com/issue/KTOR-637

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1052

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

Java 1.8.0_181 Is what my JAVA_HOME is set to.

Also:

IntelliJ IDEA 2019.1 (Ultimate Edition)
Build #IU-191.6183.87, built on March 27, 2019
JRE: 1.8.0_202-release-1483-b39 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.3

Feedback

I'm seeing the following error in the browser console (chrome), while serving the page locally on my computer: Failed to load resource: net::ERR_INCOMPLETE_CHUNKED_ENCODING

That's all the information with the error. This is the first time I've seen this. It's just another indicator that there seems to be some race condition with the ktor code. Again, I can't reproduce this.

Here is the video that I saw the error happen with while loading. I have to zip it so github accepts it:
intersection-time-lapse.mp4.zip

The video is loaded via this html element:

    <video id="home-background-video" muted playsinline autoplay loop preload="auto" poster="assets/images/intersection-first-frame.jpg" width="1920" height="1080">
      <source src="assets/images/intersection-time-lapse.mp4" type="video/mp4">
    </video>

The video is served via this ktor server code, via the static files route:

fun main() {
    embeddedServer(
        Netty,
        port = 8080,
        module = Application::main
    ).apply { start(wait = true) }

}

private fun Application.main() {
    features()
    registration()
    staticFiles()
}

private fun Application.features() {
    install(Compression)
    install(ContentNegotiation) {
        jackson {
            configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            configure(SerializationFeature.INDENT_OUTPUT, false)
        }
    }
}

private fun Application.staticFiles() {
    routing {
        static {
            files("web")
            default("web/index.html")
        }
    }
}

SessionTransportTransformerEncrypt.kt throws java.security.InvalidAlgorithmParameterException: Wrong IV length: must be 16 bytes long when using AES 256

A slow HTTP post can block the server from processing requests

Error on call.

Bug in get method with jackson serialization

https://youtrack.jetbrains.com/issue/KTOR-525

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1445

Ktor Version and Engine Used (client or server and name)
version, client/server, additional ktor modules included

Describe the bug
Jackson cant serialize a json into a model on Get but it works on Post with the exact same code.

route("/auth") {
                get() { //throw error at runtime
                    val post = call.receive<PostUser>()
                    call.respond(HttpStatusCode.Created)
                }
                post() { //works fine
                    val post = call.receive<PostUser>()
                    call.respond(HttpStatusCode.Created)
                }
            }

Expected behavior
Ability to serialize body json in get method.

By sending in postman:
GET http://0.0.0.0:8080/users/auth
in raw json

{
	"user": {
		"id": "767e5d91-1f84-4761-9fe6-1efe176d0d7a",
		"username": "typescripst",
		"password": "strddingentz"
	}
}

I get the following error (unlike with post for the same request)

2019-11-17 00:15:30.104 [nioEventLoopGroup-4-1] ERROR Application - Unhandled: GET - /users/auth
com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (InputStreamReader); line: 1, column: 0]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4145)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4000)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3049)
	at io.ktor.jackson.JacksonConverter.convertForReceive(JacksonConverter.kt:44)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:153)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:118)
	at com.back.ApplicationKt$module$5$2$3$1.invokeSuspend(Application.kt:109)
	at com.back.ApplicationKt$module$5$2$3$1.invoke(Application.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:98)
	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:180)
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:97)
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:137)
	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:106)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:36)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:26)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:20)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:515)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

ps:

I've installed jackson with

install(DataConversion)
install(ContentNegotiation) {
        jackson {
            enable(SerializationFeature.INDENT_OUTPUT)
        }
    }

io.ktor.sessions.CacheStorage.invalidate() rethrows cached exception

CIO server html generation problem. Missing port

Support the server to run on Android API <=25

Accessing custom config does not work with "-config" option

No response after 16k requests using siege

Reading from CIO's call.request.receiveChannel() causes 100% CPU usage

Feature request - Challenge Builder Function for DigestAuth

Enabling and disabling TSL protocols

Compression support stream

Stopping the server prints errors.

https://youtrack.jetbrains.com/issue/KTOR-718

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/758

When stopping the server in this test for my custom feature to handle single page applications strange errors shows up and idk ho to handle them:

17:36:27.091 [Test worker @coroutine#1] INFO ktor.application - No ktor.deployment.watch patterns specified, automatic reload is not active
17:36:27.123 [Test worker @coroutine#1] INFO ktor.application - Responding at http://127.0.0.1:8080
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol init
INFORMAZIONI: Initializing ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector
INFORMAZIONI: Using a shared selector for servlet write/read
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardService startInternal
INFORMAZIONI: Starting service [Tomcat]
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardEngine startInternal
INFORMAZIONI: Starting Servlet Engine: Apache Tomcat/9.0.10
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol start
INFORMAZIONI: Starting ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol pause
INFORMAZIONI: Pausing ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardService stopInternal
INFORMAZIONI: Stopping service [Tomcat]
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol stop
INFORMAZIONI: Stopping ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol destroy
INFORMAZIONI: Destroying ProtocolHandler ["http-nio-8080"]
BUILD SUCCESSFUL in 3s
7 actionable tasks: 5 executed, 2 up-to-date
17:36:28: Tasks execution finished ':cleanTest :test --tests "it.lamba.ktor.features.Tests.testResources"'.