This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1623

Ktor Version and Engine Used (client or server and name)
1.3.0, client

Describe the bug
If response contains "Location" and status 302
So according to https://en.wikipedia.org/wiki/HTTP_302
the request type of the new request change to GET

But in the current version the method of a new request is still the same as an original request

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1894

Ktor Version and Engine Used (client or server and name)
Using io.ktor:ktor-client-ios:1.3.2

Describe the bug
This code prints kotlin.IllegalStateException: [generateNonce] is not supported on iOS:

  GlobalScope.launch {
    try {
      HttpClient {
        install(WebSockets)
      }.wss(
        host = "websocketstest.com",
        path = "/service"
      ) {
        send(Frame.Text("timer"))
        for (frame in incoming)
          println((frame as Frame.Text).readText())
      }
    } catch (e: Exception) {
      println(e)
    }
  }

Looks like someone tried to fix this here: https://github.com/ktorio/ktor/pull/1535

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/556

Since client is MPP now, it'll be great if websocket support also ported to MPP. Now it's jvm-only since http-cio is jvm module.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1215

Guys, Do you have any plans to support ktor-http-cio/posix/src/io/ktor/http/cio/websocket?

There is the nice one library https://github.com/korlibs/korio, thanks Carlos.
But now this library is not supported for 1.3.30+ kotlin. And I think we should use best api for this one, like in ktor.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1967

Subsystem
Client/Server, particular relevant ktor module(s) if applicable.
ktor HTTP-client, auth

Is your feature request related to a problem? Please describe.
Currently, Ktor HTTP-client supports only 2 types of providers: Basic and Digest. But in a real-life project is a standard to use Bearer token pair - access and refresh. So it will be great to add support of Bearer tokens to multiplatform Ktor's HTTP-client.
So the problem itself can be split into 4 sub-tasks:

1. Persist token pair in a safe place
2. Add them to auth headers
3. In case when the request with access token returns 401,
   3.1. pause all present requests
   3.2 refresh it with refresh token.. and so on, regular flow.
   3.3 restart interrupted requests from 3.1

Describe the solution you'd like
Create BearerAuthConfig, BearerAuthProvider like its Basic and Digest versions. Update Auth class to be able to save, load and refresh tokens. Something like this:

fun provide(): HttpClient = HttpClient {
        install(Auth) {
            bearer {
                refresh = TODO("lambda function that get new access token and return a string"),
                revoke = TODO("lambda function that removes tokens")
            }
        }
    }

Motivation to include to ktor
Add support of Bearer tokens for clients.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1285

Subsystem
Client

Is your feature request related to a problem? Please describe.
Sometimes external service returns 500/502 errors, however, next call can be successful. In this case, do connection retry after a small delay. Similar, a timeout or network errors could be fixed by a retry.

Describe the solution you'd like
I'd like to have separate Ktor Client feature to automate these retries. For example, it can get handler like needRetry: Response -> Bool on input and configurable retry pauses and retry count.

Motivation to include to ktor
Of course, retries can be implemented above the call. However, the client can do them better, because:

• It can preserve low-level resources (like data buffers, resolved headers, etc.)
• It already does retry logic on 401 responses

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1840

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-client-websockets:$ktor_version")
implementation("io.ktor:ktor-client-cio:$ktor_version")

Describe the bug
A program using ktor-client-cio can't connect to localhost via 0.0.0.0. Instead, a ConnectTimeoutException is thrown.

To Reproduce

• Checkout the 0-0-0-0-cio-connection-issue branch of the repro project
• Run the included EchoServer.kt. This will start a local server that answers websocket requests on :9090/chat with a short text fragment.
• Run the included Simplified.kt. This makes a webSocket request inside a runBlocking call in order to retrieve the text fragment from the server.
• Instead of retrieving the text snippet, an exception is thrown:

Exception in thread "main" io.ktor.network.sockets.ConnectTimeoutException: Connect timeout has been expired [url=ws://0.0.0.0:9090/chat, connect_timeout=unknown ms]
	at io.ktor.client.features.HttpTimeoutKt.ConnectTimeoutException(HttpTimeout.kt:157)
	at io.ktor.client.features.HttpTimeoutKt.ConnectTimeoutException$default(HttpTimeout.kt:156)
	at io.ktor.client.engine.cio.Endpoint.getTimeoutException(Endpoint.kt:213)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:205)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:93)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior
The WebSocket should connect successfully (like it does when using 127.0.0.1 instead of 0.0.0.0).

If you switch the websocket implementation from ktor-client-cio to ktor-client-okhttp, the client behaves as expected (ignoring my previously reported termination issue)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1924

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/593

Whenever I try and do a simple http request on macOs the program freezes and reports a memory leak:

an example where I publish the latest eap13 branch commit to localMaven which is tagged (v0.9.5-rc13):

buildscript {
    repositories {
        jcenter()
        maven { url 'https://plugins.gradle.org/m2/' }
        maven { url 'https://dl.bintray.com/jetbrains/kotlin-native-dependencies' }
        maven {url 'http://dl.bintray.com/kotlin/kotlin-dev'}
    }
    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:0.9.2-dev-4008"
    }
}

allprojects {

    project.ext {
        //make extras set inside buildscript available in all modules
        kotlin_version = '1.3.0-rc-51'
        kotlin_coroutines_version = '0.26.1-eap13'
    }
}


apply plugin: 'kotlin-platform-native'

repositories {
    mavenLocal()
    jcenter()
}



components.main {
    targets = ["macos_x64"]
    outputKinds = [EXECUTABLE]
    dependencies {
        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core-native:0.26.1-eap13")


        //you need to clone ktor and publish to local maven
        implementation "io.ktor:ktor-client-ios:0.9.4-SNAPSHOT"
    }

}

example program:

import kotlinx.coroutines.runBlocking
import io.ktor.client.engine.ios.*
import io.ktor.client.*
import io.ktor.http.*
import io.ktor.client.request.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import platform.darwin.*

internal val ApplicationDispatcher: CoroutineDispatcher = NsQueueDispatcher(dispatch_get_main_queue())

fun main(args: Array<String>) = runBlocking<Unit> {
    val api = ApplicationApi()
    api.about {
        println(it)
    }
    delay(3000L)

    return@runBlocking
}

internal class NsQueueDispatcher(
        private val dispatchQueue: dispatch_queue_t
) : CoroutineDispatcher() {
    override fun dispatch(context: CoroutineContext, block: Runnable) {
        dispatch_async(dispatchQueue) {
            block.run()
        }
    }
}

class ApplicationApi {
    private val client = HttpClient()

    fun about(callback: (String) -> Unit) {
        launch(ApplicationDispatcher) {
            val result: String = client.get {
                url {
                    protocol = URLProtocol.HTTPS
                    port = 443
                    host = "tools.ietf.org"
                    encodedPath = "rfc/rfc1866.txt"
                }
            }

            callback(result)
        }
    }
}

when run:

/opt/teamcity-agent/work/4d622a065c544371/runtime/src/main/cpp/Memory.cpp:1124: runtime 
assert: Memory leaks found
Abort trap: 6

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/537

What I expect:

val http = HttpClient(Apache.create()) {
  install(HttpBasePath) {
    basePath = "http://somewhere.in"
  }
}
http.get("/a") // => http://somewhere.in/a
http.get("http://a.b.com") // => http://a.b.com

I tried implementing the above feature, but I couldn't tell a "localhost" string is the something set or just default.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/684

My team is trying to move from using OkHttp to using ktor as our HttpClient. We've found this difficult because there doesn't seem to be a way to set the default base url.

If we have the following requests:

https://www.example.com/api/getA/
https://www.example.com/api/getB/
https://www.example.com/api/getC/

It's common practice to have the ability to set the base url for the http client as https://www.example.com/api/, and then only specify the suffix of the path (/getA/, /getB/, /getC/) at the actual request site.

I like the way you can create a defaultRequest when creating the http client with ktor and then intuitively override what's needed at the request site, however this functionality needs to be extended to allow the inclusion of path data in the base url. This functionality is available in OkHttp and also Retrofit. Otherwise it requires significant refactoring in our code-base when switching between environments, as base urls and base paths often change between environments. I imagine this will be a common problem.

If somehow this is already possible, it's not clear that it is, and in which case, this should be documented better.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1970

https://github.com/ktorio/ktor/blob/3ce3906c2bfa3c86238d40e9b67006b2b020bdaf/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L43

I'd like to change this to internal so I can consume the boundary like so:

class MultiPartMixedDataContent(
        parts: List<PartData>
) : MultiPartFormDataContent {
    override val contentType: ContentType = ContentType.MultiPart.Mixed.withParameter("boundary", boundary)
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1708

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-client-core-jvm:1.3.1")
implementation("io.ktor:ktor-client-core:1.3.1")
implementation("io.ktor:ktor-client-jackson:1.3.1")
implementation("io.ktor:ktor-client-logging-jvm:1.3.1")
implementation("io.ktor:ktor-client-okhttp:1.3.1")
implementation("io.ktor:ktor-jackson:1.3.1")
implementation("io.ktor:ktor-metrics-micrometer:1.3.1")
implementation("io.ktor:ktor-metrics:1.3.1")
implementation("io.ktor:ktor-server-host-common:1.3.1")
implementation("io.ktor:ktor-server-netty:1.3.1")

Describe the bug

We've started seeing sporadic OkHttp exceptions in our tests when we upgraded from 1.3.0 to 1.3.1. Upon downgrading back to 1.3.0 (with no other changes) the tests were fine again.

The exception is:

java.io.EOFException: \n not found: limit=0 content=…
	at o.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:240)
	at o.i.h.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242)
	at o.i.h.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:213)
	... 20 common frames omitted
Wrapped by: java.io.IOException: unexpected end of stream on http://called-service/...
	at o.i.h.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:236)
	at o.i.c.Exchange.readResponseHeaders(Exchange.java:115)
	at o.i.h.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.c.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at o.i.c.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at o.i.h.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at o.i.h.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
	at o.RealCall$AsyncCall.execute(RealCall.java:172)
	at o.i.NamedRunnable.run(NamedRunnable.java:32)
	at j.u.c.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at j.u.c.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:834)

It looks like the HTTP client receives an EOF while reading the response headers from calling http://called-service/... I don't think the culprit is the remote web server though, because this server is called from other services and it's only the one with ktor 1.3.1 that throws these errors. And this one is also fine with ktor 1.3.0.

This issue has already been raised before in the OkHttp repository, for example here.

From what I can see, both ktor 1.3.0 and 1.3.1 both depend on com.squareup.okhttp3:okhttp:3.14.2, so maybe this is due to a change in the way ktor uses OkHttp?

To Reproduce

Not easy to reproduce, unfortunately, because the error only occurs in a small percentage of requests (~5%). I'm hoping someone else observes this issue and can come up with a reproducible test.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1570

Subsystem
Client

Is your feature request related to a problem? Please describe.
I want to make http requests to XML API.

Describe the solution you'd like
Create ktor-client-xml like ktor-client-json to (de-) serialize request/response.

Motivation to include to ktor
XML very popular format for data transferring.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1223

Ktor Version

1.2.2

Ktor Engine Used(client or server and name)

ktor-client-apache

JVM Version, Operating System and Relevant Context

jvm: 1.8.1_171, OS: Windows

Feedback

Run this code:

import io.ktor.client.HttpClient
import io.ktor.client.engine.apache.Apache
import io.ktor.client.request.get
import kotlinx.coroutines.async
import kotlinx.coroutines.runBlocking
import org.junit.Test

class MemoryLeak {

    @Test
    fun `memory leak`() {
        runBlocking {
            async {
                while (true) {
                    httpClient.get<String>("http://google.com")
                }
            }
        }
    }

    private val httpClient = HttpClient(Apache) {
        engine {
            customizeClient {
                setMaxConnPerRoute(1000)
                setMaxConnTotal(1000)
            }
        }
    }
}

Then invoke following script:

jmap -histo:live <pid> |  grep Invoke

you will get something like:

  14:          1307          52280  kotlinx.coroutines.InvokeOnCancelling
  ...

Wait for a while, invoke script again, you will find that instance count are continuously increasing.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1042

Ktor Version

1.2.0-alpha-2

Ktor Engine Used(client or server and name)

Client

JVM Version, Operating System and Relevant Context

ktor-client-logging

Feedback

On non HttpStatusCode 200 even while using sl4j or a logging system that supports large content buffers, it seems like large response bodies is not correctly handled but gets stuck with "BODY START" and then several minutes later shows "A resource failed to call response.body().close()."

In this particular case the error response is a json object

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1695

Ktor Version and Engine Used (client or server and name)
Ktor Client Common 1.3.1, in browser (JS)

Describe the bug
request to /example requests http://localhost/example

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/Ribesg/Kita/tree/2c7941183ebcde1fc2193bd00a1bed53f1249b10 on a machine which can be reached from the network (a server)

   git clone https://github.com/Ribesg/Kita
   cd Kita
   git reset --hard 2c7941183ebcde1fc2193bd00a1bed53f1249b10
   

2. Run ./runServer.sh
3. Browse to https://yourserver:12345
4. Try to login using whatever credentials (correct ones are admin/admin)
5. See failing requests to https://localhost/api/auth/login instead of https://yourserver:12345/api/auth/login in the console

Expected behavior
Requests should use the current location.origin when none is provided in the browser

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1257

Hi,

I'm trying to force the cache of some responses, that have no cache headers, using ktor client.

In okhttp client i can add an interceptor and change the response headers to force the response to be cached.

I tried to do the same in ktor, creating an feature, adding an interceptor to the receiving pipeline, but i'm not finding an way to override the headers on the response.
Is there an way to do it?

I also tried making a custom cache feature, based on the HttpCache provided, but most of the classes needed to that are internal, an due to that i could not achieve the result i needed.

Thanks in advance

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1304

Subsystem
Locations

Is your feature request related to a problem? Please describe.
When writing a multiplatform frontend (i.e. Android, iOS) using ktor-client, one has to use strings for get() / update() / delete() calls to a (Ktor-based) backend. This is tedious and error-prone and replicates what had to tbe done for the backend.

Describe the solution you'd like
ktor-locations is available for clients as well and can be used in multiplatform projects.

Motivation to include to ktor
Clients using ktor-client would benefit from using the same type-safe API path types that can be used for backends. This would increase code re-use and allows for easier refactoring.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1309

Ktor Version and Engine Used (client or server and name)
1.2.3

Describe the bug
I get this exception when i try to do a lot of request concurrently. Same code works with the Apache Client.

To Reproduce

HttpClient(CIO).use { client ->
            metadata.keys.forEach {
                val meta = jackson.writeValueAsString(metadata.getValue(it))
                client.call {
                    url("$baseUrl/api/matrices/create")
                    method = HttpMethod.Post
                    body = TextContent("""{"name": "$it", "repository":"GE_VIENNA", "rows":10, "metaData": $meta}""", ContentType.Application.Json)
                }.close()
            }
        }

Expected behavior
No exception

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/895

Hey!

I'm using ktor in my multiplatform application. I have problem with testing ktor client get request to any page. It totally freezes. Other tests work. (Same test on Android works well)

My test looks like this:

@Test
fun downloadPageContent() {
    runBlocking(Dispatchers.Unconfined) {
        HttpClient().use {
            it.get<String>("https://stackowerflow.com")
        }
    }
}

My run ios test gradle script:

task iosTest {
    description = "Runs tests for target 'ios' on an iOS simulator"

    def device = project.findProperty("iosDevice")?.toString() ?: "iPhone 8"
    dependsOn 'linkTestDebugExecutableIos'
    group = JavaBasePlugin.VERIFICATION_GROUP

    doLast {
        def binary = kotlin.targets.ios.compilations.test.getBinary('EXECUTABLE', 'DEBUG')
        exec {
            commandLine 'xcrun', 'simctl', 'spawn', device, binary.absolutePath
        }
    }
}

Versions:

ktor_version = '1.1.1'
coroutines_version = '1.1.0'
kotlin_version = '1.3.11'

PS:
If i change url to random characters error no longer prints, but it still freezes.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1839

Ktor Version and Engine Used (client or server and name)

implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("io.ktor:ktor-client-websockets:$ktor_version")
implementation("io.ktor:ktor-client-okhttp:$ktor_version")

Describe the bug
A program using ktor-client-okhttp does not terminate properly even after closing a WebSocket. Instead, it just hangs at the end of the main method.

To Reproduce
Steps to reproduce the behavior:
Check out the repro repo at https://github.com/SebastianAigner/reproduce-ktor-client-coroutine-issue
Run Simplified.kt (or App.kt).
Console output is as follows:

Entering blocking websocket part...
RECV: PING
Done!

At this point, the program gets stuck. Debugger shows that it is stuck in java.lang.ref.Reference#waitForReferencePendingList.

Expected behavior
The program should terminate after printing "Done!". Note that if you switch out the WebSocket implementations from

implementation("io.ktor:ktor-client-okhttp:$ktor_version")

to

implementation("io.ktor:ktor-client-cio:$ktor_version")

The program terminates as expected – which is why this bug seems to be limited to the okhttp implementation.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1813

Ktor Version and Engine Used (client or server and name)
1.3.2, client: apache (but same behavior on CIO), server: netty, enabled calls logging
Http client:

fun httpClient() =
    HttpClient(Apache) {
        install(JsonFeature) {
            serializer = JacksonSerializer()
        }

        install(ResponseObserver) {
            onResponse {
                httpClientLogger.trace { "Response received" }
            }
        }

        install(Logging) {
            logger = Logger.TRACE
            level = LogLevel.ALL
        }
    }

Describe the bug
I have following code:

        runCatching {
            client.get<ConversationInformation> {
                url(endpoint)
                header("Authorization", "Bearer $token")
            }
        }.onFailure {
            logger.error(it) { "It was not possible to fetch conversation information!" }
        }.onSuccess {
            logger.debug { "Successfully got conversation information." }
            logger.trace { it }
        }.getOrNull()

And when executed, SOMETIMES it never logs either onFailure nor onSuccess block.
The client GET request in some cases either silently fails or gets stuck/suspended for indefinite time, not sure how is that possible, but following logs are produced by the call logging mentioned previously.

21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - REQUEST: https://<internal URL>
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Authorization: Bearer <some bearer>
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Accept: application/json
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - -> Accept-Charset: UTF-8
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - CONTENT HEADERS
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: null
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging -
21/04/2020 12:27:54 [TRACE] com.wire.HttpCallsLogging - BODY END
21/04/2020 12:27:55 [TRACE] com.wire.ObserverLogger - Response received
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: application/json
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - RESPONSE: 200 OK
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - FROM: https://<internal URL>
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Date: Tue, 21 Apr 2020 12:27:54 GMT
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Content-Type: application/json
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Vary: Accept-Encoding
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Content-Length: 125
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Via: 1.1 google
21/04/2020 12:27:55 [TRACE] com.wire.HttpCallsLogging - -> Alt-Svc: clear

It seems like the body was never received.

When the request is successful, log looks like this:

21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - REQUEST: https://<internal URL>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Authorization: Bearer <some bearer>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Accept: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Accept-Charset: UTF-8
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - CONTENT HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: null
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging -
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY END
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Response received
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Sending to registry
21/04/2020 12:43:24 [TRACE] com.wire.ObserverLogger - Registered
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY Content-Type: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - BODY START
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - RESPONSE: 200 OK
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - METHOD: HttpMethod(value=GET)
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - FROM: https://<internal URL>
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - COMMON HEADERS
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Date: Tue, 21 Apr 2020 12:43:24 GMT
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Content-Type: application/json
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Vary: Accept-Encoding
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Content-Length: 125
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Via: 1.1 google
21/04/2020 12:43:24 [TRACE] com.wire.HttpCallsLogging - -> Alt-Svc: clear
21/04/2020 12:43:24 [DEBUG] c.w.b.p.services.ConversationService - Successfully got conversation information.
21/04/2020 12:43:24 [TRACE] c.w.b.p.services.ConversationService - ConversationInformationd(dto=SomeDataHere)
21/04/2020 12:43:25 [TRACE] com.wire.HttpCallsLogging - {"id":"3f7fad1d-5066-487d-8836-9e40928bb12f","name":"a","members":[{"id":"420e4b22-fac2-47e8-b07f-63958d7fcccd","status":0}]}
21/04/2020 12:43:25 [TRACE] com.wire.HttpCallsLogging - BODY END

To Reproduce
It is happening for like 1/5 of all requests, whole service code available here

Expected behavior
IF the request is not successful, the client should throw exception or eventually time out and then throw exception and not to silently die. Not sure what is happening though

EDIT:
tried with

            val response = client.get<HttpStatement> {
                url(endpoint)
                header("Authorization", "Bearer $token")
            }.execute()
            logger.trace { "Executed" }

as well and if it gets stuck, Executed is never printed. Also, seems similar to #679

EDIT2:
I found out that the flow is following:

1. http client executes GET
2. target server respond with 200
3. http client gets stuck
4. target server throws SocketTimeoutException after a while

I this case I would expect exception to be thrown.

EDIT3:
this is definitely connected to

        install(ResponseObserver) {
            onResponse {
                httpClientLogger.trace { "Response received" }
            }
        }

as without observer, everything is nice and shiny

EDIT4:
Happens with Jetty and OkHttp engines as well.
Mostly it happens after multiple requests, like first one or two works, but then it stops working. This is happening in our staging Kubernets.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1745

Ktor Version and Engine Used (client or server and name)
1.3.2, client Curl or IOS

Describe the bug
IllegalStateException is thrown when acceptContentTypes and contentType are different.
When both are same, the client call is ok and http response is correct.

To Reproduce
Steps to reproduce the behavior:

@Serializable
data class User(
    val email: String,
    val password: String
)

@UnstableDefault
@KtorExperimentalAPI
class Sample {
    val client: HttpClient by lazy {
        HttpClient(Curl.create()) {
            expectSuccess = false

            install(JsonFeature) {
                acceptContentTypes = listOf(
                    ContentType.parse("application/vnd.any.response+json")
                )
                serializer = KotlinxSerializer()
            }
        }
    }
}

@ImplicitReflectionSerializer
@KtorExperimentalAPI
@UnstableDefault
fun main() = runBlocking {
    val response: HttpResponse = Sample().client.put {
        url("https://httpbin.org/put")
        contentType(
            ContentType.parse("application/vnd.any+json")
        )
        body = Json(JsonConfiguration.Stable).toJson(User("mail@mail.com", "password1234"))
    }

    println(response.readText())
}

Uncaught Kotlin exception: kotlin.IllegalStateException: Fail to send body. Content has type: class kotlinx.serialization.json.JsonObject, but OutgoingContent expected.
        at 0   untitled.kexe                       0x00000001002e6317 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37)
        at 1   untitled.kexe                       0x00000001002df5d5 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
        at 2   untitled.kexe                       0x00000001002df775 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
        at 3   untitled.kexe                       0x00000001002dfc35 kfun:kotlin.IllegalStateException.<init>(kotlin.String?)kotlin.IllegalStateException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:70:44)
        at 4   untitled.kexe                       0x00000001005d0206 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invokeSuspend#internal + 5270 (/Users/teamcity3/buildAgent/work/4d622a065c544371/backend.native/build/stdlib/kotlin/util/Preconditions.kt:98:15)
        at 5   untitled.kexe                       0x00000001005d0728 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invoke#internal + 312 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpSend.kt:72:71)
        at 6   untitled.kexe                       0x0000000100549d22 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1122 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:207:0)
        at 7   untitled.kexe                       0x000000010054937b kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:18:0)
        at 8   untitled.kexe                       0x00000001005ccdec kfun:io.ktor.client.features.HttpRequestLifecycle.Feature.$install$lambda-0COROUTINE$25.invokeSuspend#internal + 1212 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpRequestLifecycle.kt:34:21)
        at 9   untitled.kexe                       0x00000001005cd508 kfun:io.ktor.client.features.HttpRequestLifecycle.Feature.$install$lambda-0COROUTINE$25.invoke#internal + 312 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpRequestLifecycle.kt:28:73)
        at 10  untitled.kexe                       0x0000000100549d22 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1122 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:207:0)
        at 11  untitled.kexe                       0x000000010054937b kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:18:0)
        at 12  untitled.kexe                       0x00000001005497da kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal + 474 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:183:16)
        at 13  untitled.kexe                       0x0000000100543909 kfun:io.ktor.util.pipeline.Pipeline.execute(TContext;TSubject)TSubject + 361 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/Pipeline.kt:27:41)
        at 14  untitled.kexe                       0x00000001005a44ed kfun:io.ktor.client.HttpClient.$executeCOROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 653 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:164:25)
        at 15  untitled.kexe                       0x00000001005a4834 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:163:13)
        at 16  untitled.kexe                       0x00000001005e0c0f kfun:io.ktor.client.statement.HttpStatement.$executeUnsafeCOROUTINE$96.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 943 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:104:27)
        at 17  untitled.kexe                       0x00000001005e0f1b kfun:io.ktor.client.statement.HttpStatement.executeUnsafe$ktor-client-core()io.ktor.client.statement.HttpResponse + 235 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:101:22)
        at 18  untitled.kexe                       0x00000001005dfbbb kfun:io.ktor.client.statement.HttpStatement.$executeCOROUTINE$93.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 1259 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:43:38)
        at 19  untitled.kexe                       0x00000001005e0584 kfun:io.ktor.client.statement.HttpStatement.execute(kotlin.coroutines.SuspendFunction1<io.ktor.client.statement.HttpResponse,T>){0<kotlin.Any?>}Generic + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:42:13)
        at 20  untitled.kexe                       0x00000001005e06f4 kfun:io.ktor.client.statement.HttpStatement.execute()io.ktor.client.statement.HttpResponse + 228 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:58:43)
        at 21  untitled.kexe                       0x000000010028c905 kfun:sample.$main$lambda-0COROUTINE$0.invokeSuspend#internal + 6117 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/statement/HttpStatement.kt:71:32)
        at 22  untitled.kexe                       0x00000001003068dc kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 700 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:26:0)
        at 23  untitled.kexe                       0x0000000100467d0a kfun:kotlinx.coroutines.DispatchedTask.run() + 2570 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:42:0)
        at 24  untitled.kexe                       0x000000010044241d kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()kotlin.Long + 813 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:272:20)
        at 25  untitled.kexe                       0x00000001004748f8 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 1864 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:65:44)
        at 26  untitled.kexe                       0x0000000100473aaf kfun:kotlinx.coroutines.runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>){0<kotlin.Any?>}Generic + 1231 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50:22)
        at 27  untitled.kexe                       0x0000000100473fca kfun:kotlinx.coroutines.runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>;kotlin.Int){0<kotlin.Any?>}Generic + 330 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:33:8)
        at 28  untitled.kexe                       0x000000010028aff2 kfun:sample.main() + 162 (/Users/niccheva/code/kotlin/untitled/src/macosMain/kotlin/sample/SampleMacos.kt:47:14)
        at 29  untitled.kexe                       0x000000010028e06a Konan_start + 138 (/Users/niccheva/code/kotlin/untitled/src/macosMain/kotlin/sample/SampleMacos.kt:47:1)

Expected behavior
Expected client call to works when accept and content type are different.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1696

Ktor Version and Engine Used (client or server and name)
Client 1.3.1, JVM 11

Describe the bug
Sometimes Ktor Http client raises ClosedSelectorException for tcp connectivity issues. Looks like (I'm not sure) this error is the result of overloaded server, which opens tcp channels for too long time.

To Reproduce
Unfortunately, there aren't exact steps, which lead to error.

0. Prepare overloaded slow server on nginx (limit tcp connections also).
1. Construct Ktor Client.
2. Execute the following:

val response = client.request<HttpResponse>(url)

if (status.isSuccess()) {
    return ClientResponse.Success(response.readText())
}
return null

Expected behavior
String with response is returned OR error with clear text/type is raised (IOException, TimeoutCancellationException, etc.)

Actual behavior
The following error is raised sometimes:

java.nio.channels.ClosedSelectorException
	at io.ktor.network.selector.ActorSelectorManager.publishInterest(ActorSelectorManager.kt:145)
	at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.kt:47)
	at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:33)
	at io.ktor.network.sockets.TcpSocketBuilder.connect(Builders.kt:107)
	at io.ktor.client.engine.cio.ConnectionFactory.connect(ConnectionFactory.kt:26)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invokeSuspend(Endpoint.kt:159)
	at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invoke(Endpoint.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:78)
	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:57)
	at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:167)
	at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:93)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:561)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:727)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:667)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:655)

So from API user it is unable to proper understand, what was happened. Moreover, this error shows some integrity fails (not sure), however only official wrappers were used (see readText function)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1395

Ktor Version and Engine Used (client or server and name)
Common:
using
apply plugin: 'kotlinx-serialization'

dependencies:
io.ktor:ktor-client-core:1.2.5
io.ktor:ktor-client-json:1.2.5
io.ktor:ktor-client-serialization:1.2.5
org.jetbrains.kotlinx:kotlinx-serialization-runtime-common:0.13.0
Android:
io.ktor:ktor-client-android:1.2.5
io.ktor:ktor-client-json-jvm:1.2.5
io.ktor:ktor-client-serialization-jvm:1.2.5

Describe the bug
Requesting multiple times in a short period (e.g. concurrent calls) from the library causes a crash.

To Reproduce
Steps to reproduce the behavior:

1. Creating a button to load any url, click it multiple times quickly, eventually it will fail
2. Using K/N and kodein

bind<HttpClient>() with provider {
        HttpClient {
            install(JsonFeature) {
                serializer = KotlinxSerializer(Json.nonstrict).apply {
                    setMapper(ApiResponse::class, ApiResponse.serializer())
                }

            }
            install(HttpCallValidator)
        }
    }

And to consume it:

override suspend fun getResponse(): ApiResponse  {
        val apiResponse = client.get<ApiResponse> {
            apiUrl("random")
        }
        return ApiResult.FetchedReponse(apiResponse)
    }
}

3. See error

Process: com.vonderful.testapi.lib, PID: 19866
    io.ktor.client.call.NoTransformationFoundException: No transformation found: class kotlinx.coroutines.io.ByteBufferChannel (Kotlin reflection is not available) -> class com.vonderful.testapi.data.ApiResponse (Kotlin reflection is not available)
    at io.ktor.client.call.HttpClientCall.receive(HttpClientCall.kt:88)
    at com.vonderful.testapi.data.RemoteApiDataSourceImpl.getRandom(RemotePhotoApiDataSource.kt:48)
    at com.vonderful.testapi.data.RemoteApiDataSourceImpl$getRandom$1.invokeSuspend(Unknown Source:11)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:215)
    at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:172)
    at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
    at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)

Expected behavior
Even with a debounce system of 500 the io.ktor.client.call.NoTransformationFoundException: No transformation found: class kotlinx.coroutines.io.ByteBufferChannel (Kotlin reflection is not available) is quite easy to reproduce...

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1265

Ktor Version and Engine Used (client or server and name)
1.2.3

Describe the bug
We are trying to create our project on kotlin multiplatform. Now trying to create architecture. Firstly I was getting 415 from server. After I changed content-type I started to get this error.

java.lang.String cannot be cast to io.ktor.client.call.HttpClientCall

It does not crash and no more logs included. I am using okhttp client. Before setting content-type, I could see the logs of my request. But after setting it I do not see those logs only the error above. It looks like it does not even make request because of this error.

To Reproduce
gradle file

kotlin {
    targets {
        fromPreset(presets.jvm, 'android')
    }

    sourceSets {
        commonMain.dependencies {
            api 'org.jetbrains.kotlin:kotlin-stdlib-common'

            implementation "io.ktor:ktor-client:1.2.3"
            implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-common:1.0.1"
            implementation "io.ktor:ktor-client-json:1.2.3"
            implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.11.0"
        }

        androidMain.dependencies {
            api 'org.jetbrains.kotlin:kotlin-stdlib'

            implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.11.0"
            implementation "io.ktor:ktor-client-serialization-jvm:1.2.3"
            implementation 'io.ktor:ktor-client-okhttp:1.2.3'
            implementation "com.squareup.okhttp3:logging-interceptor:4.0.1"
            implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.0.1"
        }
    }
}

engine

actual val clientEngine = HttpClient(OkHttp){
    install(JsonFeature) {
        serializer = KotlinxSerializer()
    }
    engine {
        val loggingInterceptor = HttpLoggingInterceptor()
        loggingInterceptor.level = HttpLoggingInterceptor.Level.BODY
        addInterceptor(loggingInterceptor)
    }

}.engine

request

suspend fun <T> makeRequest(requestMethod: HttpMethod, requestPath: String, requestModel: SerializationStrategy<T>, obj: T): HttpResponse {
        val json = Json(JsonConfiguration.Stable)
        val bodyStr = json.stringify(requestModel, obj)

        return client.request {
            url {
                protocol = URLProtocol.HTTPS
                method = requestMethod
                host = baseUrl
                encodedPath = requestPath
                body = bodyStr
                headers {
                    header("Content-Type", "application/json")
                }
            }
        }
    }

Expected behavior
not giving error and making request

If you need more info let me know.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1226

Ktor Version and Engine Used (client or server and name)

implementation "io.ktor:ktor-client-core:1.2.2"
implementation "io.ktor:ktor-client-websockets:1.2.2"
implementation "io.ktor:ktor-client-okhttp:1.2.2"

Describe the bug
Calling WebSocketSession.terminate() on OkHttp WebSocket session takes a while to terminate. Even worse, OkHttp keeps non-daemon thread alive for this, which prevents program from stopping in that period.

To Reproduce
Run following code:

fun main() = runBlocking {
    val client = HttpClient {
        install(WebSockets)
    }

    val session = client.webSocketSession {
        url("ws://some-websocket-API")
    }

    session.terminate()
}

Expected behavior

Documentation states that

Initiate connection termination immediately. Termination may complete asynchronously.

So I assume that above example code should execute and then terminate immediately, but it takes a while to do so.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1184

Ktor Version 1.2.0

###Ktor Engine Used(client or server and name) Client OkHttp
###JVM Version, Operating System and Relevant Context JVM 1.8

install(HttpCallValidator) {
            validateResponse { response ->
                val statusCode = response.status.value
                when (statusCode) {
                    var responseString = response.readText //DoubleReceiveException
                    in 400..499 -> throw  TheCustomExceptionIWant(parseIntoObject(responseString))
                   ...
                }
            }
        }

Hi (⌒_⌒;), I'm a bit new to Kotlin. I wanted to parse the custom json errors returned from our server.
ex.

    error : {
           title : "some error"
    }
    code : "422" 
}```

I've always thought the HttpCallValidator feature is a good way to do this instead of putting handlers in each api calls with JsonFeature - Kotlinx Serializer. 

How do I achieve reading the response from the validator?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1180

The following URL causes a redirect:
https://www.facebook.com/pg/253802271688277/

In Chrome and when using curl the redirect URL is decoded like this:
https://www.facebook.com/GRAÇA-77-253802271688277/

Ktor client interprets it differently. There the value of the Location header looks like this:
https://www.facebook.com/GRAÇA-77-253802271688277/

And after passing it to Url(…) like this:
https://www.facebook.com/GRA%C3%83%C2%87A-77-253802271688277/

I'd expect it to be interpreted just like by the other clients.

The URL sent by Facebook is UTF-8 encoded. However the Apache client by default simply converts every single byte to char without further decoding.

The following configuration makes the redirect work properly.
Maybe it can be considered a good default for a more predictable behavior.

HttpClient(Apache) {
   engine {
      customizeClient {
         setDefaultConnectionConfig(ConnectionConfig.custom().setCharset(Charsets.UTF_8).build())
      }
   }
}

Ktor Version

1.2.0

Ktor Engine Used(client or server and name)

Apache client

JVM Version, Operating System and Relevant Context

JDK 1.8.0.191, macOS

Somewhat related to #607.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1084

Ktor

ktor-client-android 1.1.4

Context

Java 1.8
Kotlin 1.3.30
Gradle 5.4
Android Gradle Plugin 3.4.0
Multiplatform context
Android app: no ProGuard nor R8, Multidex enabled

Implementation

MPP Library H exposes a factory of HttpClient
MPP Library M implements the Algolia REST API using library H
Non-MPP Android app uses library M to call Algolia's search API

Result

Only difference is the Android version. Both requests are done roughly at the same time on two different emulators running the same APK, and running on the same MacOS machine.

• On Android Q, the request works as expected, returns status code 200 and the expected json
• On Android L, the request returns a status code 400 with a basic json result:

  {
    "message": "Bad request",
    "status": 400
  }
  

The experience could be reproduced again and again. 100% 200 on Android Q and 100% 400 on Android L. Restarting and wiping both devices had no impact. Complete clear and rebuild of project had no impact.
Further investigation with Algolia shows that the request never actually reached them: there's no trace of a 400 response on the server side. The lack of standard Algolia headers in the 400 response seem to confirm this.
The requests are using HTTPS.

I had to switch to ktor-client-okhttp and move on. Didn't have time to create dozens on emulators to test which version of Android works and which doesn't.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1029

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

ktor-client-cio

JVM Version, Operating System and Relevant Context

Android 4.2.2

Feedback

Getting crash that is not passed to the higher level. I think its related to TLS support on old android versions.

03-19 23:36:30.471 26546-26546/com.example.android.kotlincoroutines E/AndroidRuntime: FATAL EXCEPTION: main
    java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
        at org.apache.harmony.xnet.provider.jsse.TrustManagerImpl.checkTrusted(TrustManagerImpl.java)
        at org.apache.harmony.xnet.provider.jsse.TrustManagerImpl.checkServerTrusted(TrustManagerImpl.java)
        at io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:207)
        at io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:146)
        at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(TLSClientHandshake.kt)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
        at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1461

Ktor Version and Engine Used (client or server and name)
Android client, version 1.2.4

Describe the bug
When using the android client in a library inside an Android application, the build fails with the error:

More than one file was found with OS independent path 'META-INF/kotlinx-io.kotlin_module'

The workaround is to add

packagingOptions {
    exclude 'META-INF/*.kotlin_module'
 }

to the applications build.gradle.

The problem is that I'm the distributor of the library and not the applications and trying to migrate from Retrofit to Ktor. Therefore, I can't ask the applications to do that.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/940

Ktor Version

1.1.2

Ktor Engine Used(client or server and name)

CIO (currently the only available WebSocket client provided by Ktor)

JVM Version, Operating System and Relevant Context

Java 11, Linux

Feedback

Currently, as far as I could find, there is apparently no convenient way in the Ktor WebSocket client API to specify the subprotocol(s) to connect with. I'm talking about the value of the "Sec-WebSocket-Protocol" header. Ktor provides a way to specify this when you set up a WebSocket server, through the protocol argument in Route.webSocket(). On the client side however, there is no such argument. I would expect HttpClient.ws and HttpClient.wss to have a similar "protocol" argument as well, but they don't.

As a workaround you can explicitly pass on an HttpRequestBuilder instance to those functions as the "request" argument, in which you then explicitly set the "Sec-WebSocket-Protocol" header (available as the value HttpHeaders.SecWebSocketProtocol in the Ktor API), but as soon as you do so, this overrides any HTTP authentication that you installed in the HttpClient by invoking install(Auth). So subsequently, you then also have to manually encode the username and password in the case of basic authentication. I expect that you'd have to jump through even more hoops in the case of HTTP Digest authentication.

Ideally, I would like to see an optional protocol (String?) argument added to the HttpClient.ws and HttpClient.wss functions to take the "Sec-WebSocket-Protocol" stuff out of our hands, while still being compatible with the install(Auth) function when configuring the HttpClient.

Below is code that reproduces the issue I'm describing here. If you comment out the "header(HttpHeaders.Authorization, ...)" part, you'll see that the "install(Auth) { basic { ... } }" stuff will be ignored when performing a WebSocket upgrade. Be sure to change the "host" and "path" argument to point to some actually running WebSocket server (and if necessary change "client.wss" to "client.ws") to run this code.

import io.ktor.client.HttpClient
import io.ktor.client.features.auth.Auth
import io.ktor.client.features.auth.providers.basic
import io.ktor.client.features.websocket.WebSockets
import io.ktor.client.features.websocket.wss
import io.ktor.client.request.header
import io.ktor.http.HttpHeaders
import io.ktor.http.HttpMethod
import io.ktor.http.cio.websocket.Frame
import io.ktor.http.cio.websocket.readText
import kotlinx.coroutines.channels.filterNotNull
import kotlinx.coroutines.channels.map
import kotlinx.coroutines.runBlocking
import java.util.Base64

val httpBasicAuthName = "username"
val httpBasicPassword = "password"

private val client = HttpClient().config {

    // NOTE: install(Auth) doesn't actually work when passing on a custom request argument to client.wss().
    install(Auth) {
        basic {
            username = httpBasicAuthName
            password = httpBasicPassword
        }

        install(WebSockets)
    }

}

runBlocking {
    client.wss(
        method = HttpMethod.Get,
        host = "somedomain.com",
        path = "/some-endpoint",
        request = {

            // See https://www.iana.org/assignments/websocket/websocket.xml#subprotocol-name
            header(HttpHeaders.SecWebSocketProtocol, "ocpp2.0,ocpp1.6")

            /* NOTE: Using install(Auth) in HttpClient().config doesn't work here for HTTP Basic Authentication,
             * since we're passing in this explicit request lambda argument, because we need to specify the
             * "Sec-WebSocket-Version" header.
             */
            header(
                HttpHeaders.Authorization,
                "Basic ${Base64.getEncoder().encodeToString(
                    "$httpBasicAuthName:$httpBasicPassword".toByteArray(
                        Charsets.UTF_8
                    )
                )}"
            )

        }) {
        // this: DefaultClientWebSocketSession
        send(Frame.Text("Hello World"))

        for (message in incoming.map { it as? Frame.Text }.filterNotNull()) {
            println(message.readText())
        }
    }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1139

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Client, Apache

JVM Version, Operating System and Relevant Context

OpenJDK 11.0.3, Ubuntu 18.04
Communicating with an external enterprise-grade server

Feedback

The remote server that our customer uses to provide services returns the Expires cookie in a wrong, non-standard date format. We do not have the power to convince them to change this behaviour.

It would be nice if we can intercept the "wrong" cookie and fix the formatting ourselves, so that https://github.com/ktorio/ktor/blob/68eef1948ce4c8c11dae794588f8d063afb67d0b/ktor-http/common/src/io/ktor/http/DateUtils.kt#L20 doesn't blow up.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1665

Ktor Version and Engine Used (client or server and name)
Ktor Version: 1.3.1
Ktor Client: Android and iOS
Ktor Modules: serialization, logging

Describe the bug
When sending a GET request the body changes unexpected passing from NoContent to "{}". This is making the request to fail.

To Reproduce
Steps to reproduce the behavior:

protected val httpClient = HttpClient(Engine) {
        install(JsonFeature) {
            serializer = KotlinxSerializer(Json.nonstrict)
        }
        install(Logging) {
            loggingConfig = this
            logger = Logger.SIMPLE
            level = LogLevel.ALL
        }
        defaultRequest {
            header(HttpHeaders.CacheControl, "no-cache")
            header(HttpHeaders.ContentType, "application/json; charset=utf-8")
            header(HttpHeaders.Accept, "application/json")
            header(HttpHeaders.AcceptEncoding, "gzip")
            header(HEADER_MWAPPS_APPID, appId)
            header(HEADER_MWAPPS_CLIENT, client)
            header(HEADER_MWAPPS_CLIENT_VERSION, clientVersion)
        }
    }
  
fun testGet(callback: Callback<String>) = CoroutineScope(Background).launch {
        val response = httpClient.get<User>(urlString = "https://jsonplaceholder.typicode.com/todos/1")
        withContext(Main) {
            callback.invoke(ApiResponse.create(response))
        }
    }

iOS error

GET method must not have a body
finished with error [-1103] Error Domain=NSURLErrorDomain Code=-1103 "resource exceeds maximum size"

Android error
Request of type GET couldn't send a body with the [Android] engine.

Expected behavior
Ktor should passed a EmptyContent and not change the content.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1829

ktorVersion = 1.3.2
implementation "io.ktor:ktor-client-core:$ktorVersion"
implementation "io.ktor:ktor-client-okhttp:$ktorVersion"
implementation "com.squareup.okhttp3:logging-interceptor:4.5.0"

sending file in form, code below works (file fitting the buff)

        val retMessage =  client.request<Message> {
            method = HttpMethod.Patch
            url("https://myhost/rest/v1/message/image-upload/aa")

            header("X-Client-Locale", settings.locale)
            header("Authorization", "Bearer ${settings.jwt}")
            userAgent(settings.userAgent)

            body = MultiPartFormDataContent(
                formData {
                    val filename = reader.open()
                    val buff = ByteArray(102400)
                    val sz = reader.read(buff)
                    if (sz > 0) {
                        append(
                            "picture",
                            filename,
                            ContentType.parse(reader.contentType),
                            size = reader.size
                        ) {
                            writeFully(buff, 0, sz)
                        }
                    }
                    reader.close()
                }
            )
        }

when I move reader.read into BytePacketBuilder

val buff = ByteArray(102400)
                    append(
                        "picture",
                        filename,
                        ContentType.parse(reader.contentType),
                        size = reader.size
                    ) {
                        val sz = reader.read(buff)
                        if (sz > 0) {
                            writeFully(buff, 0, sz)
                        }
                    }

getting:

    ---3e33c4c9bf0fe2451394e9a-21e8f6434bf41b1b-3454ee4d-35548052-6fcbb83a7a--
    --> END PATCH (26800-byte body)
D/OkHttp: <-- HTTP FAILED: java.net.ProtocolException: unexpected end of stream
E/LonjeShared: Rest.request(/rest/v1/message/image-upload/aa): java.net.ProtocolException: unexpected end of stream

same issue happens with InputProvider

                    val filename = reader.open()
                    val inp = reader.input
                    if (inp != null) {
                        append(
                            "picture",
                            InputProvider(reader.size) {
                                return@InputProvider inp
                            }
                        )
                    }

it shows appropriate HTTP PATCH content in OkHttp log

please help me find an issue, I hope I'm doing something wrong. But it can be a serious ktor issue.

PS: on serverside I'm getting:

ERRO multipart: NextPart: stream error: stream ID 7; CANCEL
INFO Mon, 27 Apr 2020 21:00:03 UTC 76.77.185.144   400 PATCH https://myhost/rest/v1/message/image-upload/aa HTTP/2.0   33B in 1.351711ms

what shows HTTP2 stream exception

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/877

Ktor Version

1.1.1

Ktor Engine Used(client or server and name)

N/A

JVM Version, Operating System and Relevant Context

N/A

Feedback

When using the DefaultRequest feature it prevents the caller being able to specify its own values.

For example if you do this:

defaultRequest {
  url {
    host = "example.com"
  }
}

all requests will always be against the host example.com even if a user does this:

client.call(Url("https://google.com/foo"))

If anything the name of the feature is misleading and should perhaps be called AllRequests instead.

This is especially hard since a lot of properties in Url have default values of their own so it's hard to know if they have been set explicitly or not.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1022

Ktor Version

1.1.13, 1.0.1

Ktor Engine Used(client or server and name)

server Netty

JVM Version, Operating System and Relevant Context

11.0.2, 8

Feedback

HttpBin example:
Sending requests to the GET /cache endpoint always results in 200 response, no matter how ETag or Last-Modified headers are set. Setting Last-Modified inside install(ConditionalHeaders) with a hard-coded date makes the server respond with 304.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1110

Ktor Version

1.2.0-rc, and previous versions as well.

Ktor Engine Used(client or server and name)

CIO client

JVM Version, Operating System and Relevant Context

OpenJDK 1.8.0_192 on Solus (Linux distribution)

Feedback

One of my projects, which uses Ktor's websockets, has a feature wherein sockets are opened and closed periodically. When a signal is received, the HttpClient that runs the sockets is closed, and all resources managed by the program itself are released. Despite this, the program does not exit. When the coroutines are dumped post-resource-release (using the coroutines debug utility), the trace contains several coroutines with no origin in any of my code, and some with an origin in my code and which originate from a wss call. Some are suspended, some are active, and it takes several minutes for all of the coroutines to stop and for the program to finally exit. Here is the full coroutines dump after all resources are cleaned up: https://hastebin.com/xevakaqulo

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1814

Ktor Version and Engine Used (client or server and name)

Version 1.3.2

My project uses ktor-client-core and ktor-client-core-js

Describe the bug
I have a React web application built with the kotlin-wrappers and ktor as the http client. When loading the application in IE 11, the application fails to load and the console displays an error saying: crypto is undefined

This problem happens in IE 11 on both windows 7 and windows 10.

#1283 references this same problem.

To Reproduce
Steps to reproduce the behavior:

1. Start web application. Application will not render because of crypto is undefined.

Expected behavior
Application should load and run without javascript exceptions from ktor.

Screenshots
If applicable, add screenshots to help explain your problem.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1333

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-client-core:1.2.4
io.ktor:ktor-client-core-jvm:1.2.4
io.ktor:ktor-client-apache:1.2.4

Describe the bug
Apache HTTP Client does not send Content-Length header if body is empty content.
CIO send the header correctly.

To Reproduce
Steps to reproduce the behavior:

1. Write the following

val apacheClient = HttpClient(Apache)
apacheClient.put<HttpResponseData>("http://example.com/") 
apacheClient.post<HttpResponseData>("http://example.com/")
apacheClient.patch<HttpResponseData>("http://example.com/")

2. Run on jvm
3. The sent header does not contain Content-Length

{
  "accept-charset": "UTF-8",
  "accept": "*/*",
  "user-agent": "Ktor client",
  "host": "example.com",
  "connection": "Keep-Alive" 
}

Some endpoints that do not require body return 411 Length Required.

Expected behavior
Content-Length: 0 Should be sent.

Apache seems not to send Content-Length if entity is null.
Following code sent the header correctly.

apacheClient.put<HttpResponseData>("http://example.com/"){
    body = ""
}

The relevant part of the code in ktor:
https://github.com/ktorio/ktor/blob/f3858b2bd95dcb35aa7755451403e7cab1dd678c/ktor-client/ktor-client-apache/jvm/src/io/ktor/client/engine/apache/ApacheRequestProducer.kt#L122-L155

and org.apache.http.client.methods.RequestBuilder 's build() method.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1832

Subsystem
Client

Is your feature request related to a problem? Please describe.
Getting how much data been transferred always useful

Describe the solution you'd like
want to get eventlistener with interface bytesSent/bytesTotal

Motivation to include to ktor
I think it already exists but I can't find it.

having code:

        val retMessage =  client.request<Message> {
            method = HttpMethod.Patch
            url("https://myhost/rest/v1/message/image-upload/aa")

            body = MultiPartFormDataContent(
...

want to be able to add something like:

        val retMessage =  client.request<Message> {
                 eventListener = { event ->
                            println("${event.bytesSent/event.bytesTotal}% uploaded")
                  }


            method = HttpMethod.Patch
            url("https://myhost/rest/v1/message/image-upload/aa")
...

I assume currently it exists but I don't know how to do it, can somebody help me please :) ?

It's deprecated and looks like TextEncoder and TextDecoder work on majority of environments.

To reproduce run the following code:

fun main() {
    val client = HttpClient(Jetty) {}
    runBlocking {
        val r = client.get<String>("http://www.google.com")
        println(r)
    }
}

As a result, org.eclipse.jetty.io.EofException will be thrown:

Exception in thread "main" org.eclipse.jetty.io.EofException
	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:283)
	at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
	at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
	at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
	at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:259)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
	at org.eclipse.jetty.http2.HTTP2Session.newStream(HTTP2Session.java:543)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$jettyRequest$1.invoke(JettyHttpRequest.kt:40)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$jettyRequest$1.invoke(JettyHttpRequest.kt)
	at io.ktor.client.engine.jetty.UtilsKt.withPromise(utils.kt:14)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt.executeRequest(JettyHttpRequest.kt:39)
	at io.ktor.client.engine.jetty.JettyHttpRequestKt$executeRequest$1.invokeSuspend(JettyHttpRequest.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.nio.channels.AsynchronousCloseException
	at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:501)
	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:263)
	... 18 more

If I change the URL to http://example.com then the app will hang indefinitely, here is a stacktrace:

"main" #1 prio=5 os_prio=0 cpu=337.00ms elapsed=56.16s tid=0x00007f8e98018800 nid=0x79547 waiting on condition  [0x00007f8e9eb77000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x00000006890f1518> (a kotlinx.coroutines.BlockingCoroutine)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:87)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at MainKt.main(main.kt:86)
	at MainKt.main(main.kt)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.36ms elapsed=56.11s tid=0x00007f8e98311000 nid=0x79550 waiting on condition  [0x00007f8e80382000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.7/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@11.0.7/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.7/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=1.63ms elapsed=56.11s tid=0x00007f8e98315000 nid=0x79551 in Object.wait()  [0x00007f8e80281000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.7/Native Method)
	- waiting on <0x00000006890f2008> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006890f2008> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.7/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.22ms elapsed=56.10s tid=0x00007f8e9832a000 nid=0x79552 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 cpu=497.80ms elapsed=56.10s tid=0x00007f8e9832c000 nid=0x79553 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=306.59ms elapsed=56.10s tid=0x00007f8e9832e000 nid=0x79554 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #7 daemon prio=9 os_prio=0 cpu=6.70ms elapsed=56.09s tid=0x00007f8e98330000 nid=0x79555 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #8 daemon prio=8 os_prio=0 cpu=0.37ms elapsed=56.03s tid=0x00007f8e98381800 nid=0x79556 in Object.wait()  [0x00007f8e47efd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.7/Native Method)
	- waiting on <0x00000006890f27e0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.7/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006890f27e0> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@11.0.7/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)
	at jdk.internal.misc.InnocuousThread.run(java.base@11.0.7/InnocuousThread.java:134)

"Monitor Ctrl-Break" #9 daemon prio=5 os_prio=0 cpu=8.39ms elapsed=55.91s tid=0x00007f8e98695800 nid=0x79557 runnable  [0x00007f8e479d4000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(java.base@11.0.7/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.7/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:140)
	at sun.nio.cs.StreamDecoder.readBytes(java.base@11.0.7/StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(java.base@11.0.7/StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(java.base@11.0.7/StreamDecoder.java:178)
	- locked <0x00000006890f4d00> (a java.io.InputStreamReader)
	at java.io.InputStreamReader.read(java.base@11.0.7/InputStreamReader.java:185)
	at java.io.BufferedReader.fill(java.base@11.0.7/BufferedReader.java:161)
	at java.io.BufferedReader.readLine(java.base@11.0.7/BufferedReader.java:326)
	- locked <0x00000006890f4d00> (a java.io.InputStreamReader)
	at java.io.BufferedReader.readLine(java.base@11.0.7/BufferedReader.java:392)
	at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:61)

"Service Thread" #10 daemon prio=9 os_prio=0 cpu=0.04ms elapsed=55.91s tid=0x00007f8e98696800 nid=0x79558 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"ktor-jetty-dispatcher-worker-1" #11 daemon prio=5 os_prio=0 cpu=230.54ms elapsed=55.68s tid=0x00007f8e989de000 nid=0x7955a waiting on condition  [0x00007f8e46ec0000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:357)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.park(CoroutineScheduler.kt:783)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.tryPark(CoroutineScheduler.kt:728)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:711)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

"ktor-jetty-dispatcher-worker-2" #12 daemon prio=5 os_prio=0 cpu=9.27ms elapsed=55.68s tid=0x00007f8e989df000 nid=0x7955b waiting on condition  [0x00007f8e46dbf000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:357)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.park(CoroutineScheduler.kt:783)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.tryPark(CoroutineScheduler.kt:728)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:711)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

"ktor-jetty-client-qtp-13" #13 prio=5 os_prio=0 cpu=6.95ms elapsed=55.47s tid=0x00007f8e340da000 nid=0x7955d runnable  [0x00007f8e46575000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPoll.wait(java.base@11.0.7/Native Method)
	at sun.nio.ch.EPollSelectorImpl.doSelect(java.base@11.0.7/EPollSelectorImpl.java:120)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@11.0.7/SelectorImpl.java:124)
	- locked <0x000000069f414300> (a sun.nio.ch.Util$2)
	- locked <0x000000069f4142a8> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(java.base@11.0.7/SelectorImpl.java:141)
	at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:472)
	at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:409)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
	at org.eclipse.jetty.io.ManagedSelector$$Lambda$65/0x0000000800176c40.run(Unknown Source)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-14" #14 prio=5 os_prio=0 cpu=126.50ms elapsed=55.47s tid=0x00007f8e340dc000 nid=0x7955e waiting on condition  [0x00007f8e46474000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-15" #15 prio=5 os_prio=0 cpu=0.27ms elapsed=55.47s tid=0x00007f8e340dd800 nid=0x7955f waiting on condition  [0x00007f8e46373000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069fc076c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.7/AbstractQueuedSynchronizer.java:2211)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:309)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:379)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-16" #16 prio=5 os_prio=0 cpu=2.68ms elapsed=55.46s tid=0x00007f8e340df800 nid=0x79560 waiting on condition  [0x00007f8e46272000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-17" #17 prio=5 os_prio=0 cpu=0.10ms elapsed=55.46s tid=0x00007f8e340e1000 nid=0x79561 waiting on condition  [0x00007f8e46171000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-18" #18 prio=5 os_prio=0 cpu=0.07ms elapsed=55.46s tid=0x00007f8e340e3000 nid=0x79562 waiting on condition  [0x00007f8e46070000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-19" #19 prio=5 os_prio=0 cpu=0.07ms elapsed=55.46s tid=0x00007f8e340e5000 nid=0x79563 waiting on condition  [0x00007f8e45f6f000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"ktor-jetty-client-qtp-20" #20 prio=5 os_prio=0 cpu=0.06ms elapsed=55.46s tid=0x00007f8e340e7000 nid=0x79564 waiting on condition  [0x00007f8e45e6e000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f418170> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.7/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.7/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"Scheduler-779376001-1" #21 prio=5 os_prio=0 cpu=10.86ms elapsed=55.42s tid=0x00007f8e34117000 nid=0x79565 waiting on condition  [0x00007f8e45b6d000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x000000069f415c20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.7/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.7/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.7/ScheduledThreadPoolExecutor.java:1170)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.7/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.7/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.7/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.7/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

"Attach Listener" #22 daemon prio=9 os_prio=0 cpu=8.56ms elapsed=55.08s tid=0x00007f8e4805c800 nid=0x79567 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=16.20ms elapsed=56.11s tid=0x00007f8e98309000 nid=0x7954f runnable  

"GC Thread#0" os_prio=0 cpu=22.88ms elapsed=56.16s tid=0x00007f8e98041800 nid=0x7954a runnable  

"GC Thread#1" os_prio=0 cpu=22.20ms elapsed=55.66s tid=0x00007f8e5c001000 nid=0x7955c runnable  

"G1 Main Marker" os_prio=0 cpu=0.51ms elapsed=56.15s tid=0x00007f8e980cc800 nid=0x7954b runnable  

"G1 Conc#0" os_prio=0 cpu=0.07ms elapsed=56.14s tid=0x00007f8e980ce000 nid=0x7954c runnable  

"G1 Refine#0" os_prio=0 cpu=0.50ms elapsed=56.13s tid=0x00007f8e9823b800 nid=0x7954d runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=10.66ms elapsed=56.13s tid=0x00007f8e9823d800 nid=0x7954e runnable  
"VM Periodic Task Thread" os_prio=0 cpu=42.21ms elapsed=55.91s tid=0x00007f8e98698000 nid=0x79559 waiting on condition  

JNI global refs: 14, weak refs: 0

If I make the same requests with curl: curl --http2-prior-knowledge http://example.com then I get an understandable error:

curl: (16) Error in the HTTP2 framing layer

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1930

Subsystem
Client, JVM

Describe the solution you'd like
In Java 11 was introduced HTTP Client based on reactive-streams.

Motivation to include to ktor
Since more projects are switching to modern Java versions it could be valuable to use built-in Java HTTP Client without need to use additional libraries. It supports HTTP/1.1 and HTTP/2 protocol and provides async APIs.

I am using Ktor client to upload images view MultiPartFormDataContent. I had the uploads working fine on Android with Retrofit and Alamofire on iOS. I have been using Ktor for a quite a while for many other APIs with no issues. Though this is the first time using multipart form uploads.

I traced the issue to the MultiPartFormDataContent adding two line feeds at the start of the writeTo method, which results in a response like so:

...
Connection: Keep-Alive
Accept-Encoding: gzip



---16a03a2d-3da73e118462adf-1f57a65d76b8040a-4b1915e2-70e2a0874eea586452
Content-Disposition: form-data; name="p_comId"
...

Note the extra lines separating the headers from the body. This breaks the server I am currently using (Apache/2.4.41 () mod_owa 2.11.8) which I cannot modify. The server returns a 400 error and does not process the request.

I worked around it by copy/pasting the MultiPartFormDataContent code into a local file with a different name and commenting out these two lines (lines 109-110):

channel.writeFully(RN_BYTES)
channel.writeFully(RN_BYTES)

which then fixed the error. The code which is using it like so:

                        body = MultiPartFormDataContentCustom(
                            formData {
                                append("\"p_comId\"", "${auth.companyId}")
                               ...
                            })

An aside: perhaps this is what is causing the other two issues I see here in YouTrack related to MultiPartFormDataContent?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1690

Ktor Version and Engine Used (client or server and name)
version: 1.3.1
lib: ktor-client-cio
additional modules: ktor-client-logging-jvm

Describe the bug
Content-type header is missing when LogLevel.BODY is set. (also I am using multi-part body, not sure if relevant). Header exists if removing logging or setting LogLevel.HEADER.

To Reproduce
Steps to reproduce the behavior:

1. Set logging

HttpClient {
  install(Logging) {
  logger = Logger.ANDROID
  level = LogLevel.BODY
  }
}

2. Do multi-form request

val response = httpClient.post<String> {
  url("http://someurl")
  body = MultiPartFormDataContent(formData {
    append(
          "file",
          "file.txt",
          ContentType.parse("text/plain"),
          contentBytes.size.toLong()
    ) {
      writeFully(contentBytes)
    }
})

3. Observe missing Content-Type: multipart/form-data; <...> header

Expected behavior
Content-Type: multipart/form-data; <...> in request exists

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1892

Ktor Version and Engine Used (client or server and name)
1.3.2, client, io.ktor.client.features.cache.HttpCache

Describe the bug
If a server returns a blank or an invalid Expires HTTP header, the ktor client crashes if the feature HttpCache is enabled install(HttpCache)

To Reproduce
Steps to reproduce the behavior:

1. Write the following:

val articleUrl = Url("https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503")
val newsClient = HttpClient(Apache) {
    install(HttpCache)

    install(UserAgent) {
        agent = "Mozilla/5.0"
    }
}
try {
    val call: HttpStatement = newsClient.get(articleSimilarityRequest.articleUrl)
    val response: HttpResponse = call.receive()
    val requestUrl = response.request.url.toString()
    val rawHtml = response.readText()
    println(rawHtml)
} catch (cause: Throwable) {
    println("Unable to download article")
} finally {
    newsClient.close()
}

2. If you open the URL https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503 in the browser, you can see that the Expires response header is blank
3. An exception is thrown if you run the code:

2020-05-23 14:41:24.124 [DefaultDispatcher-worker-2 @request#48] ERROR a.s.api.util.service.DownloadService - Unable to download article
java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	(Coroutine boundary)
	at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:123)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:116)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:89)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
	at app.shwoo.api.util.service.DownloadService.downloadArticle$suspendImpl(DownloadService.kt:101)
	at app.shwoo.api.server.web.SimilarityResourceKt$similarity$1$4.invokeSuspend(SimilarityResource.kt:38)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
	at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:292)
	at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
	at io.ktor.server.testing.TestApplicationEngine$handleRequest$pipelineJob$1.invokeSuspend(TestApplicationEngine.kt:290)
Caused by: java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior
It seems that the problem occurs in https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/features/cache/HttpCacheEntry.kt#L84. If the Expires header is blank an exception is thrown. The expected behavior is that the application handles the cache as expired if an invalid date is provided in the HTTP Expires header. Like described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires

Make the message of ResponseException contain response's text from response.readText().

The following URL

https://data.smartdublin.ie/cgi-bin/rtpi

returns valid results with other HTTP clients, including IntelliJ HTTP Requests. However with Ktor, independently of engine, it throws a parser exception (side note, it would be good to indicate the header in the exception result)

Exception in thread "main" io.ktor.http.cio.ParserException: Character with code 1 is not allowed in header names, 
HTTP/1.1 200 OKServer: nginx/1.17.8Date: Sun, 19 Jul 2020 07:12:42 GMTContent-Type: application/json; charset=utf-8Content-Length: 1596054Connection: keep-aliveX-Powered-By: ASP.NETSet-Cookie: visid_incap_1822411=6ycr4UzsSv+6sdm+QnNmpmnyE18AAAAAQUIPAAAAAACm9JYEnYu4a0fTA7VxKyxu; expires=Sun, 18 Jul 2021 08:40:35 GMT; HttpOnly; path=/; Domain=.nationaltransport.ieSet-Cookie: incap_ses_535_1822411=vjDoPwoqvTdguklauLNsB2nyE18AAAAAoieq1zxaD46bWhzEJyQd3w==; path=/; Domain=.nationaltransport.ieSet-Cookie: ___utmvmzauvysSB=WQYQExBjiZv; path=/; Max-Age=900Set-Cookie: ___utmvazauvysSB=Tjh
nJRz; path=/; Max-Age=900
	at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:279)
	at io.ktor.http.cio.HttpParserKt.parseHeaderValue(HttpParser.kt:261)
	at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:119)
	at io.ktor.http.cio.HttpParserKt.parseResponse(HttpParser.kt:73)
	at io.ktor.http.cio.HttpParserKt$parseResponse$1.invokeSuspend(HttpParser.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Process finished with exit code 1

To reproduce see attached code.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/997

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

OkHttp with GsonSerializer

JVM Version, Operating System and Relevant Context

Android 9.0

Feedback

When I try to do a POST call with a body object, it gives me a ClassCastException:

java.lang.ClassCastException: be.gekrabbel.ktortest.MyCustomObject cannot be cast to io.ktor.client.call.HttpClientCall
        at io.ktor.client.HttpClient.execute(HttpClient.kt:151)
        at io.ktor.client.call.HttpClientCallKt.call(HttpClientCall.kt:80)
        at be.gekrabbel.ktortest.MainActivity.callDoesntWork(MainActivity.kt:63)
        at be.gekrabbel.ktortest.MainActivity$onCreate$1.invokeSuspend(MainActivity.kt:41)

It works if the body is a String instead of an object, and it also works after explicitly adding contentType(ContentType.Application.Json).

Full sample app: https://github.com/Arne517/ktor-test/blob/master/app/src/main/java/be/gekrabbel/ktortest/MainActivity.kt

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1411

Ktor Version and Engine Used (client or server and name)
ktor 1.2.5, client, coroutines 1.3.2

Describe the bug
httpClientCall.cancel() does not work as expected.
HttpClientCall can only be canceled within the current job.
calling httpClient.Call.cancel() within another job/coroutine does not work.

To Reproduce
Create two async jobs, one has httpClientCall.receive() running, other one calls httpClientCall.cancel()

Expected behavior
httpClientCall.cancel() should cancel the job that the http request runs in

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1246

Subsystem
Engine for HttpClient - ApacheEngine

Is your feature request related to a problem? Please describe.
It seems like it's not possible to create HttpClient with Apache engine with Tracing component inside.

Describe the solution you'd like
It will be nice to construct HttpClient with custom implementation of HttpAsyncClientBuilder inside and use tracing feature implicitly.

Motivation to include to ktor
All api with clientBuilder is internal and there is no way to override it without support of ktor client.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1437

Ktor 1.2.4 - Client

• OkHttp
• JSON
• Serialization

Describe the bug
It seems that it doesn't catch 404's in an error

To Reproduce

1. Here's my code:

val httpClientEngine by lazy {
    OkHttp.create {
        if (BuildConfig.DEBUG) {
            addInterceptor(HttpLoggingInterceptor().apply {
                level = HttpLoggingInterceptor.Level.BODY
            })
        }
    }
}

private val httpClient = HttpClient(httpClientEngine) {
    install(JsonFeature) {
        serializer = KotlinxSerializer(Json.nonstrict)
    }
}

suspend fun myFunction(data: Any) = httpCall {
    httpClient.call("my_invalid_url") {
        contentType(ContentType.Application.Json
        method = HttpMethod.Post
        body = MyRequest(data)
    }
}

And here's the httpCall wrapper:

suspend inline fun <T> httpCall(call: () -> T) = try {
    call()
} catch (exception: ClientRequestException) {
    val errorResponse = exception.response.receive<MyErrorResponse>()
    var message = errorResponse.message

    throw Exception(message)
}

However, 404's just pass through this.

Here's the response from the server:

<-- 404 Not Found http://127.0.0.1/my_invalid_url (420ms)        
Server: nginx/1.15.12                                               
Date: Wed, 13 Nov 2019 07:18:25 GMT                                 
Content-Type: text/html; charset=utf-8                              
Content-Length: 152                                                 
Connection: keep-alive                                              
X-Powered-By: Express                                               
Access-Control-Allow-Origin: *                                      
Content-Security-Policy: default-src 'none'                         
X-Content-Type-Options: nosniff                                     
<!DOCTYPE html>                                                     
<html lang="en">                                                    
<head>                                                              
<meta charset="utf-8">                                              
<title>Error</title>                                                
</head>                                                             
<body>                                                              
<pre>Cannot POST /my_invalid_url</pre>                                
</body>                                                             
</html>                                                             
<-- END HTTP (152-byte body)                                        

I'm thinking I should need to register an XML serializer? But I can't find anything in the documentation. If I'm missing something, please do point me in the right direction.

Many thanks!

Expected behavior
It should catch 404's as an error

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1018

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Apache

JVM Version, Operating System and Relevant Context

Feedback

I utilize ktor http client to communicating from service to other service, but after 10 minutes deployment my machine experience CPU 100%, any idea why ?

here's the building block that I use,

override fun sendAsync(request: HttpRequest, responseCallback: Callback<HttpResponse>) {
    GlobalScope.launch (Dispatchers.IO + CoroutineName(key ?: "")) {
      try {
        decorateRequest(request)

        val requestUrl = URL(request.url)
        val requestBody = String(request.body.asArray(), StandardCharsets.UTF_8)
        val requestMethod = HttpMethod(request.method)
        val requestContentType = createContentType(request)

        val clientResponse = httpClient!!.request<io.ktor.client.response.HttpResponse>{
          method = requestMethod
          url(requestUrl)
          body = TextContent(requestBody, requestContentType)

          request.headers.forEach {
            if (!unsafeHeaderSet.contains(it.name)) {
              header(it.name, it.value)
            }
          }
        }

        val response = HttpResponse()
        val responseBodyString = String(clientResponse.readBytes())
        response.statusCode = clientResponse.status.value
        response.bodyEncoding = "UTF-8"
        response.body = IndirectNIOBuffer(ByteBuffer.wrap(responseBodyString.toByteArray(charset("UTF-8"))), true)

        clientResponse.headers.forEach { name, value ->
          val headerValue = value.joinToString(",")

          response.addHeader(name, headerValue)
        }

        clientResponse.close()
        responseCallback.onComplete(response)
      } catch (error: Exception) {
        responseCallback.onException(error)
      }
    }
  }

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1540

Ktor Version and Engine Used (client or server and name)
kotlin_version=1.3.61
ktor_version=1.3.0-rc2
coroutines_version=1.3.3-native-mt

Describe the bug
Calling testNetworkCall() from ViewController#viewDidLoad in iOS leads to a kotlin.ClassCastException

  GlobalScope.launch {
    HttpClient { }.get {
      url {
        takeFrom("https://api.basebeta.com")
        encodedPath = "/rankings"
      }
    }
  }

Full stacktrace

Uncaught Kotlin exception: kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for AwaitContinuation(Shareable[used]){HttpResponseData=(statusCode=200 OK)}@cb3ac8. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at 0   BaseBetaCommonApi                   0x0000000111c1ac63 kfun:kotlin.Error.<init>(kotlin.String?;kotlin.Throwable?)kotlin.Error + 115
        at 1   BaseBetaCommonApi                   0x0000000111d20703 kfun:kotlinx.coroutines.CoroutinesInternalError.<init>(kotlin.String;kotlin.Throwable)kotlinx.coroutines.CoroutinesInternalError + 115
        at 2   BaseBetaCommonApi                   0x0000000111d86cba kfun:kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx-coroutines-core(kotlin.Throwable?;kotlin.Throwable?) + 938
        at 3   BaseBetaCommonApi                   0x0000000111d868b0 kfun:kotlinx.coroutines.DispatchedTask.run() + 3568
        at 4   BaseBetaCommonApi                   0x0000000111d1b908 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
        at 5   BaseBetaCommonApi                   0x0000000111da26f1 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
        at 6   BaseBetaCommonApi                   0x0000000111da9d12 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
        at 7   BaseBetaCommonApi                   0x0000000111da9efb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.invoke#internal + 59
        at 8   BaseBetaCommonApi                   0x0000000111da9f5b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.$<bridge-UNN>invoke()#internal + 59
        at 9   BaseBetaCommonApi                   0x0000000111c4fd31 WorkerLaunchpad + 177
        at 10  BaseBetaCommonApi                   0x0000000112052d89 _ZN6Worker19processQueueElementEb + 2569
        at 11  BaseBetaCommonApi                   0x0000000112053336 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 12  libsystem_pthread.dylib             0x00007fff51bfe2eb _pthread_body + 126
        at 13  libsystem_pthread.dylib             0x00007fff51c01249 _pthread_start + 66
        at 14  libsystem_pthread.dylib             0x00007fff51bfd40d thread_start + 13
Caused by: kotlin.ClassCastException: kotlin.coroutines.native.internal.CompletedContinuation cannot be cast to kotlinx.coroutines.DispatchedContinuation
        at 0   BaseBetaCommonApi                   0x0000000111c21227 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87
        at 1   BaseBetaCommonApi                   0x0000000111c1a585 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85
        at 2   BaseBetaCommonApi                   0x0000000111c1a0c5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85
        at 3   BaseBetaCommonApi                   0x0000000111c1ae65 kfun:kotlin.ClassCastException.<init>(kotlin.String?)kotlin.ClassCastException + 85
        at 4   BaseBetaCommonApi                   0x0000000111c6ba5b ThrowClassCastException + 939
        at 5   BaseBetaCommonApi                   0x0000000111d14ef8 kfun:kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(kotlin.coroutines.Continuation<#STAR>) + 216
        at 6   BaseBetaCommonApi                   0x0000000111c42400 kfun:kotlin.coroutines.native.internal.ContinuationImpl.releaseIntercepted() + 768
        at 7   BaseBetaCommonApi                   0x0000000111c42a62 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1234
        at 8   BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 9   BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 10  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 11  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 12  BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 13  BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 14  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 15  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 16  BaseBetaCommonApi                   0x0000000111e8cbf2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 930
        at 17  BaseBetaCommonApi                   0x0000000111e8c773 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1699
        at 18  BaseBetaCommonApi                   0x0000000111e8e651 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353
        at 19  BaseBetaCommonApi                   0x0000000111c42b28 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1432
        at 20  BaseBetaCommonApi                   0x0000000111d8659c kfun:kotlinx.coroutines.DispatchedTask.run() + 2780
        at 21  BaseBetaCommonApi                   0x0000000111d1b908 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
        at 22  BaseBetaCommonApi                   0x0000000111da26f1 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
        at 23  BaseBetaCommonApi                   0x0000000111da9d12 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
        at 24  BaseBetaCommonApi                   0x0000000111da9efb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.invoke#internal + 59
        at 25  BaseBetaCommonApi                   0x0000000111da9f5b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$146.$<bridge-UNN>invoke()#internal + 59
        at 26  BaseBetaCommonApi                   0x0000000111c4fd31 WorkerLaunchpad + 177
        at 27  BaseBetaCommonApi                   0x0000000112052d89 _ZN6Worker19processQueueElementEb + 2569
        at 28  BaseBetaCommonApi                   0x0000000112053336 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 29  libsystem_pthread.dylib             0x00007fff51bfe2eb _pthread_body + 126

To Reproduce
Copy and paste the code snippet from above into the common module of a multiplatform project. Run the function from ViewController#viewDidLoad in the iOS app.

Expected behavior
I expect the code to execute without throwing a ClassCastException.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1636

Ktor Version and Engine Used (client or server and name)
Ktor Client 1.3.1 in Kotlin MPP context. Engines OkHTTP and iOS Native, but not explicitly set (resolved by gradle dependency).

Describe the bug
Ktor HttpClient keeps suspending when requesting to read more than 4096 bytes from a Url, and never comes back.

To Reproduce
Steps to reproduce the behavior:

suspend fun downloadFile(url: String): ByteArray {
    val httpClient = HttpClient{
        install(Logging) {
            this.logger = Logger.SIMPLE
            this.level = LogLevel.ALL
        }
    }
    val statement = httpClient.request<HttpStatement>(url) {
        range?.let {
            header(HttpHeaders.Range, range)
        }
    }
    return statement.execute {
        val contentLength = it.contentLength()?.lowInt ?: 0
        val byteArray = ByteArray(contentLength)
        var offset = 0
        do {
            val currentRead = it.content.readAvailable(byteArray, offset, byteArray.size)
            offset += currentRead
            logger.logDebug("Download in progress, offset: ${offset}, current read ${currentRead} / ${contentLength}")
        } while (offset < contentLength)
        logger.logDebug("Download done")
        return@execute byteArray
    }
}

Execute function above with a URL that contains a file larger than 4096 bytes. During my testing I'm downloading files from Amazon S3.

Expected behavior
Expected to get a byte array containing the entire contents of the file.

Screenshots
The logging in the function above generates this result:

Download in progress, offset: 4088, current read 4088 / 355835
Download in progress, offset: 4096, current read 8 / 355835

I know that using a ByteArray is not the most ideal solution to download a file into, but I'm working in a pure-kotlin Multi Platform Project, so any Java import is a no-go without using expect/actual which means I cannot directly write to files, or use a stream of some kind.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1400

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-client-js 1.3.0-beta-1 and 1.2.5

Describe the bug
Webpack "Module not found" errors are thrown.

Logs:

  WARNING in .._imported/ktor-ktor-client-core/1.3.0-beta-1/ktor-ktor-client-core.js 12165:13-32
  Critical dependency: the request of a dependency is an expression
   @ ./kotlin/gaia.js
   @ multi ./kotlin/gaia.js source-map-support/browser-source-map-support.js
  
  WARNING in .._imported/ktor-ktor-client-core/1.3.0-beta-1/ktor-ktor-client-core.js
  Module not found: Error: Can't resolve 'ws' in 'C:\Users\Daan\code\gaia\build\js\packages_imported\ktor-ktor-client-core\1.3.0-beta-1'
   @ .._imported/ktor-ktor-client-core/1.3.0-beta-1/ktor-ktor-client-core.js
   @ ./kotlin/gaia.js
   @ multi ./kotlin/gaia.js source-map-support/browser-source-map-support.js
  
  ERROR in .._imported/ktor-ktor-io/1.3.0-beta-1/ktor-ktor-io.js
  Module not found: Error: Can't resolve 'text-encoding' in 'C:\Users\Daan\code\gaia\build\js\packages_imported\ktor-ktor-io\1.3.0-beta-1'
   @ .._imported/ktor-ktor-io/1.3.0-beta-1/ktor-ktor-io.js 20149:21-45 20217:21-45
   @ .._imported/ktor-ktor-client-json/1.3.0-beta-1/ktor-ktor-client-json.js
   @ ./kotlin/gaia.js
   @ multi ./kotlin/gaia.js source-map-support/browser-source-map-support.js

To Reproduce
Steps to reproduce the behavior:

1. Add the io.ktor:ktor-client-js dependency to a Kotlin multi-platform project
2. Run ./gradlew jsBrowserWebpack
3. See errors

When doing get<HttpResponse> to e.g. only inspect the etag header or some other response header, the status code isn't validated. It's only validated when accessing the body. However, we often don't inspect the body, but only the header. This inconsistency has caused bugs on our side because we missed bad responses.

Could you please always validate the response status code, no matter if the body is accessed or not?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1242

Ktor Version and Engine Used (client or server and name)
Ktor CIO 1.2.2 (no additional modules are used)

Describe the bug
Ktor http client is unable to download and process huge response by using the following algorithm:

client.call(url).use { call ->
    call.response.use { response ->
          do {
             val line = response.content.readUTF8Line()  // <--- this code returns "" continuously after several loops

             /*process line*/
             
          } while (line !== null)
    }
}

To Reproduce
Steps to reproduce the behavior:

1. Start get request with potential huge response (I'm trying to download Teamcity Logs, size is between 1 Mb and 10 Gb, however response content size does not matter).
2. Process lines in loop, described above.
3. After several attempts readUTF8Line returns empty line continuously. So application hangs here.

Expected behavior
readUTF8Line returns:

• Next line if available
• null if stream is closed
• Exception if stream was interrupted (for example - by server)

Probably, there is related issue: #768

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1028

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Relevant code is ktor-client-core, but bug occurs in JVM using ktor-client-okhttp.

JVM Version, Operating System and Relevant Context

java 11.0.2 2019-01-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)

javac 11.0.2

Windows

Feedback

Assume a variable, val files: List<Pair<String, ByteReadPacket>> = emptyList().
If you do this:

formData {
    files.forEachIndexed { index, (first, second) ->
        append("file$index", first) { writePacket(second) }
    }
}

You'll get an internal compilation error:

e: java.lang.IllegalStateException: Backend Internal error: Exception during code generation
Cause: Back-end (JVM) Internal error: wrong code generated
org.jetbrains.kotlin.codegen.CompilationException Back-end (JVM) Internal error: Couldn't transform method node:
invoke (Lio/ktor/client/request/forms/FormBuilder;)V:
    // annotable parameter count: 1 (visible)
    // annotable parameter count: 1 (invisible)
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 1
    LDC "receiver$0"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
...

However, if you remove the formData, say, do this:

files.forEachIndexed { index, (first, second) ->
   println("$index: $first $second")
}

Or if you don't deconstruct:

formData {
    files.forEachIndexed { index, file ->
        append("file$index", file.first) { writePacket(file.second) }
    }
}

No errors. I'm almost certain this is a Kotlin issue and not specifically a Ktor one, but this is the only context I have found the issue so I bring it to attention here.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1006

Ktor Version

1.1.3

Ktor Engine Used

Apache Client

JVM Version, Operating System and Relevant Context

Java 11, Runtime Environment 18.9 (build 11+28), Windows 10

I want to catch the java.net.ConnectException, when i send a http request without printing the stacktrace. Actually i can catch it, but the stacktrace though is printed. How can i prevent it?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/545

At least conditionally, otherwise any benchmarking session hangs after a single-shot run and prints every connection to System.out

2018-08-28 12:43:54.623 [engine-thread] ERROR Application - 200 OK: GET - /
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
        at sun.nio.ch.IOUtil.write(IOUtil.java:51)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
        at io.ktor.network.sockets.CIOWriterKt$attachForWritingDirectImpl$1$1.doResume(CIOWriter.kt:73)
        at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
        at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:150)
        at kotlinx.coroutines.experimental.io.internal.MutableDelegateContinuation.run(MutableDelegateContinuation.kt:14)
        at io.ktor.network.util.IOCoroutineDispatcher$IODispatchedTask.run(IOCoroutineDispatcher.kt)
        at io.ktor.network.util.IOCoroutineDispatcher$IOThread$run$1.doResume(IOCoroutineDispatcher.kt:73)
        at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
        at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:150)
        at kotlinx.coroutines.experimental.DispatchedContinuation.run(Dispatched.kt:14)
        at kotlinx.coroutines.experimental.EventLoopBase.processNextEvent(EventLoop.kt:136)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/769

Is it possible to set cookie store per request (as opposed to per client instance)? Would you consider adding such feature?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/236

On Server Side the Serialization is implemented in the ContentNegotiation Feature and additional Support "Features". To avoid any code duplication, it would be great if the same ContentConverter could be used in the client as well.
This would IMHO also need a common module which is used in the client as well as the server.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1263

Ktor Version and Engine Used (client or server and name)

• io.ktor:ktor-client-core:1.2.3-rc
• io.ktor:ktor-client-core-jvm:1.2.3-rc
• io.ktor:ktor-client-okhttp:1.2.3-rc

Describe the bug
See the exception below. This exception is reported by Firebase. I can not reproduce it myself although quite some users appear to get this bug. I have no idea where this happens in my code and that is why I am unable to fix it myself. It looks like some sort of timeout error.

Fatal Exception: java.net.SocketTimeoutException: timeout
       at okio.SocketAsyncTimeout.newTimeoutException + 159(SocketAsyncTimeout.java:159)
       at okio.AsyncTimeout.exit$jvm + 203(AsyncTimeout.java:203)
       at okio.AsyncTimeout$source$1.read + 163(AsyncTimeout.java:163)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read + 352(Http1ExchangeCodec.java:352)
       at okhttp3.internal.http1.Http1ExchangeCodec$UnknownLengthSource.read + 488(Http1ExchangeCodec.java:488)
       at okhttp3.internal.connection.Exchange$ResponseBodySource.read + 279(Exchange.java:279)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.cache.CacheInterceptor$cacheWritingResponse$cacheWritingSource$1.read + 158(CacheInterceptor.java:158)
       at okio.RealBufferedSource.read + 148(RealBufferedSource.java:148)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke + 97(OkHttpEngineKt.java:97)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke(OkHttpEngineKt.java:1)
       at kotlinx.coroutines.io.ByteBufferChannel.write + 1508(ByteBufferChannel.java:1508)
       at kotlinx.coroutines.io.ByteWriteChannel$DefaultImpls.write$default + 84(ByteWriteChannel.java:84)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invokeSuspend + 96(OkHttpEngineKt.java:96)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith + 33(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run + 241(DispatchedTask.java:241)
       at java.util.concurrent.ThreadPoolExecutor.runWorker + 1167(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run + 641(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run + 764(Thread.java:764)
Caused by java.net.SocketTimeoutException: Read timed out
       at java.net.SocketInputStream.socketRead0(SocketInputStream.java)
       at java.net.SocketInputStream.socketRead + 119(SocketInputStream.java:119)
       at java.net.SocketInputStream.read + 176(SocketInputStream.java:176)
       at java.net.SocketInputStream.read + 144(SocketInputStream.java:144)
       at okio.InputStreamSource.read + 102(InputStreamSource.java:102)
       at okio.AsyncTimeout$source$1.read + 159(AsyncTimeout.java:159)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.http1.Http1ExchangeCodec$AbstractSource.read + 352(Http1ExchangeCodec.java:352)
       at okhttp3.internal.http1.Http1ExchangeCodec$UnknownLengthSource.read + 488(Http1ExchangeCodec.java:488)
       at okhttp3.internal.connection.Exchange$ResponseBodySource.read + 279(Exchange.java:279)
       at okio.RealBufferedSource.read + 41(RealBufferedSource.java:41)
       at okhttp3.internal.cache.CacheInterceptor$cacheWritingResponse$cacheWritingSource$1.read + 158(CacheInterceptor.java:158)
       at okio.RealBufferedSource.read + 148(RealBufferedSource.java:148)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke + 97(OkHttpEngineKt.java:97)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1$1$1.invoke(OkHttpEngineKt.java:1)
       at kotlinx.coroutines.io.ByteBufferChannel.write + 1508(ByteBufferChannel.java:1508)
       at kotlinx.coroutines.io.ByteWriteChannel$DefaultImpls.write$default + 84(ByteWriteChannel.java:84)
       at io.ktor.client.engine.okhttp.OkHttpEngineKt$toChannel$1.invokeSuspend + 96(OkHttpEngineKt.java:96)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith + 33(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run + 241(DispatchedTask.java:241)
       at java.util.concurrent.ThreadPoolExecutor.runWorker + 1167(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run + 641(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run + 764(Thread.java:764)

To Reproduce
As the stacktrace shows no lines of my code I do not know how to reproduce it. I would really appreciate some help for this exception. I am not entirely sure if this is related to Ktor, but thought this would be the best place to report it. I have Kotlin Coroutines debug mode enabled as well.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1907

Ktor Version and Engine Used (client or server and name)

implementation "io.ktor:ktor-client-cio:$ktor_version"
implementation "io.ktor:ktor-client-json-jvm:$ktor_version"
implementation "io.ktor:ktor-client-gson:$ktor_version"
ext.ktor_version = "1.3.1"

JVM 1.8, with Kotlin v1.3.72

Describe the bug
With JSON support being enabled, it can not consume a REST API.

Example Implementation

import io.ktor.client.HttpClient
import io.ktor.client.features.json.JsonFeature
import io.ktor.client.request.get


suspend fun main() {
    val clusterJson: String = HttpClient().get("http://ubuntu:8082/v3/clusters")
    println(clusterJson)

    val client: V3Cluster = HttpClient() {
        install(JsonFeature)
    }.get("http://ubuntu:8082/v3/clusters")

}

data class V3Cluster(
        val `data`: List<Data>
)

data class Data(
        val attributes: Attributes,
        val id: String,
        val type: String
)

data class Attributes(
        val cluster_id: String
)

This fails with

Exception in thread "main" io.ktor.client.features.ClientRequestException: Client request(http://ubuntu:8082/v3/clusters) invalid: 406 Not Acceptable
	at io.ktor.client.features.DefaultResponseValidationKt$addDefaultResponseValidation$1$1.invokeSuspend(DefaultResponseValidation.kt:35)
	at io.ktor.client.features.DefaultResponseValidationKt$addDefaultResponseValidation$1$1.invoke(DefaultResponseValidation.kt)
	at io.ktor.client.features.HttpCallValidator.validateResponse(HttpCallValidator.kt:35)
	at io.ktor.client.features.HttpCallValidator$Companion$install$2.invokeSuspend(HttpCallValidator.kt:96)
	at io.ktor.client.features.HttpCallValidator$Companion$install$2.invoke(HttpCallValidator.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:273)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.client.call.HttpClientCall.receive(HttpClientCall.kt:75)
	at controllers.KafkaClusterKt.main(KafkaCluster.kt:105)
....

The first get request is processed correctly, but the second one fails with the provided stacktrace.

Expected behaviour
It should deserialize the json response into an object.

Without JSON feature, the request works fine and is producing the JSON as output. Also when using curl on the terminal, the API endpoint itself seems fine:

brandl@ubuntu:~$ curl http://ubuntu:8082/v3/clusters | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   594  100   594    0     0  84857      0 --:--:-- --:--:-- --:--:-- 84857
{
  "links": {
    "self": "http://rest-proxy:8082/v3/clusters",
    "next": null
  },
  "data": [
    {
      "id": "crn:///kafka=CVyJNKWhSrmXmnRUKWo5cg",
      "links": {
        "self": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg"
      },
      "attributes": {
        "cluster_id": "CVyJNKWhSrmXmnRUKWo5cg"
      },
      "relationships": {
        "controller": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/brokers/1"
          }
        },
        "brokers": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/brokers"
          }
        },
        "topics": {
          "links": {
            "related": "http://rest-proxy:8082/v3/clusters/CVyJNKWhSrmXmnRUKWo5cg/topics"
          }
        }
      },
      "type": "KafkaCluster"
    }
  ]
}

Not relevant for the ticket, but maybe some context is helpful: I'm trying to cusome the confluent broker config API v3 as described under https://github.com/confluentinc/kafka-rest/issues/676

I get these exceptions in the log with ktor 1.3.0, but with 1.3.2 it crashes my app:

2020-09-16 10:39:40.942 30696-30937/com.thomashabets.simapprover I/Process: Sending signal. PID: 30696 SIG: 9

I'm using WebSockets and some URL fetching, nothing fancier than that.

I'm unclear on what's going on here. I'm new to kotlin, and to app development, so it could be that I'm doing something wrong. But since nothing in the stack trace is mine, I don't know where to start. It's possible that there's a race condition I'm failing to protect against.

Or is this a bug in ktor? Because exceptions in ktor code, that don't seem catchable, and this different behavior between 1.3.0 and 1.3.2, I'm suspecting it might be, and therefore am creating this ticket.

My ugly source code is here.

For now I'm staying with 1.3.0 because at least it doesn't crash my app, but only manifests in log pollution.

2020-09-16 10:39:40.941 30696-30948/com.thomashabets.simapprover E/AndroidRuntime: FATAL EXCEPTION: ktor-cio-dispatcher-worker-3
    Process: com.thomashabets.simapprover, PID: 30696
    java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:421)
        at io.ktor.utils.io.nio.ChannelsKt.read(Channels.kt:135)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invokeSuspend(CIOReader.kt:90)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invoke(Unknown Source:10)
        at io.ktor.network.util.UtilsKt.withSocketTimeout(Utils.kt:20)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:80)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:175)
        at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:137)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:306)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:316)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:248)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:84)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:64)
        at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:73)
        at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(Unknown Source:12)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
2020-09-16 10:39:40.942 30696-30980/com.thomashabets.simapprover E/AndroidRuntime: FATAL EXCEPTION: ktor-cio-dispatcher-worker-3
    Process: com.thomashabets.simapprover, PID: 30696
    java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:421)
        at io.ktor.utils.io.nio.ChannelsKt.read(Channels.kt:135)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invokeSuspend(CIOReader.kt:90)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invoke(Unknown Source:10)
        at io.ktor.network.util.UtilsKt.withSocketTimeout(Utils.kt:20)
        at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:80)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:175)
        at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:137)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:306)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:316)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:248)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:84)
        at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:64)
        at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:73)
        at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(Unknown Source:12)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1119

Ktor Version

1.2.0-rc

Ktor Engine Used(client or server and name)

Client/CIO

JVM Version, Operating System and Relevant Context

OpenJDK 11.0.3, Arch Linux.

Feedback

When the internet is disconnected, a vague java.io.IOException is thrown twice, and it cannot be caught.

Exception in thread "pool-2-thread-3" java.io.IOException: Connection reset by peer
	at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
	at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
	at kotlinx.io.nio.ChannelsKt.read(Channels.kt:117)
	at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:75)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.ResumeModeKt.resumeMode(ResumeMode.kt:67)
	at kotlinx.coroutines.DispatchedKt.resume(Dispatched.kt:309)
	at kotlinx.coroutines.DispatchedKt.resumeUnconfined(Dispatched.kt:49)
	at kotlinx.coroutines.DispatchedKt.dispatch(Dispatched.kt:295)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:250)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:260)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:189)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:83)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:63)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:74)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/360

Currently the HttpAsyncResponseConsumer in use is the internal ApacheResponseConsumer and this doesn't seem to be overridable.

For custom behaviours such as limiting the total size of the response payload without needing to actually fetch everything beforehand, would be advantageous

The HttpClient doesn't give me a chance to check thrown exceptions and directly cancels any parent coroutine instead, which results in some weird race conditions.

fun Application.module() {  
    val client = HttpClient(Apache)  
    routing {
	    get {
		    val deferred = async { client.get<String>("Some erroneous request") }
		    try {
			    call.respondText(deferred.await())
			} catch (e: Exception) {
				call.respondText("Error") //I will be cancelled
			}
		}
	}
}

As a workaround I have added a SupervisorJob to the async coroutine builder for now.

Using the Ktor MPP client to perform network calls fails in the browser with the message

Fail to fetch
captureStack@repro/build/js/node_modules/kotlin/kotlin/builtins.kt:105:376
Error_0@repro/build/js/packages/smartcrypt-sdk-client-test/adapter-browser.js:67528:26
repro/build/js/node_modules/ktor-ktor-client-core-jsLegacy/ktor-ktor-client-core-jsLegacy.js:168119
promiseReactionJob@[native code]

We've encountered this on:

• Mac + Safari, Firefox, Firefox headless
• Windows + Firefox, Firefox headless, Chrome, Chrome headless

The same code works fine for the JVM target, and also for NodeJS. The issue appears with both the legacy and the new IR backends.

Versions:

• Kotlin: 1.4.0-rc
• Ktor: 1.3.2-1.4.0-rc

In the attached repro.zip project there is a class named NetworkTest in the commonTest sourceset. Running the test it contains reproduces the issue for us.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1224

First I would very appreciate for the ktor framework, but I have a little problem of my usage. I compile the kotlin code to JS. but in the generated JS code it contain this require("ktor-ktor-http") code. So I searched ktor-ktor-http in the npm repositories, but unfortunately I can not find it in the the npm repositories. So, did I missed something?

this is my generated JS code

function (root, factory) {
  if (typeof define === 'function' && define.amd)
    define(['exports', 'kotlin', 'kotlinx-coroutines-core', 'ktor-ktor-client-core', 'ktor-ktor-http'], factory);
  else if (typeof exports === 'object')
    factory(module.exports, require('kotlin'), require('kotlinx-coroutines-core'), require('ktor-ktor-client-core'), require('ktor-ktor-http'));
  else {
    if (typeof kotlin === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'kotlin' was not found. Please, check whether 'kotlin' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['kotlinx-coroutines-core'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'kotlinx-coroutines-core' was not found. Please, check whether 'kotlinx-coroutines-core' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['ktor-ktor-client-core'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'ktor-ktor-client-core' was not found. Please, check whether 'ktor-ktor-client-core' is loaded prior to 'kb-mutiplatform'.");
    }
    if (typeof this['ktor-ktor-http'] === 'undefined') {
      throw new Error("Error loading module 'kb-mutiplatform'. Its dependency 'ktor-ktor-http' was not found. Please, check whether 'ktor-ktor-http' is loaded prior to 'kb-mutiplatform'.");
    }
    root['kb-mutiplatform'] = factory(typeof this['kb-mutiplatform'] === 'undefined' ? {} : this['kb-mutiplatform'], kotlin, this['kotlinx-coroutines-core'], this['ktor-ktor-client-core'], this['ktor-ktor-http']);
  }

this is my build.gradle

plugins {
    id 'kotlin-multiplatform' version '1.3.40'
}
repositories {
    maven {
        url 'http://maven.aliyun.com/nexus/content/groups/public/'
    }
    maven {
        url 'http://maven.aliyun.com/nexus/content/repositories/jcenter'
    }
    mavenLocal()
    mavenCentral()

    maven { url 'https://plugins.gradle.org/m2/' }
    maven { url 'https://dl.bintray.com/kotlin/kotlin-eap' }
    jcenter()
}
group 'com.example'
version '0.0.1'

apply plugin: 'maven-publish'

kotlin {
    jvm()
    js {
        compilations.main.kotlinOptions.moduleKind = 'umd'
    }
    // For ARM, should be changed to iosArm32 or iosArm64
    // For Linux, should be changed to e.g. linuxX64
    // For MacOS, should be changed to e.g. macosX64
    // For Windows, should be changed to e.g. mingwX64
//    macosX64("macos")
    iosX64("ios") {
        binaries {
            framework()
        }
    }
    sourceSets {
        commonMain {
            dependencies {
                implementation ("io.ktor:ktor-client-core:$ktor_version")
                implementation kotlin('stdlib-common')
            }
        }
        commonTest {
            dependencies {
                implementation kotlin('test-common')
                implementation kotlin('test-annotations-common')
            }
        }
        jvmMain {
            dependencies {
                implementation ("io.ktor:ktor-client-android:$ktor_version")
                implementation kotlin('stdlib-jdk8')
            }
        }
        jvmTest {
            dependencies {
                implementation kotlin('test')
                implementation kotlin('test-junit')
            }
        }
        jsMain {
            dependencies {
                implementation kotlin('stdlib-js')
                implementation "io.ktor:ktor-client-js:$ktor_version"
            }
        }
        jsTest {
            dependencies {
                implementation kotlin('test-js')
            }
        }
        iosMain {
            dependencies {
                implementation ("io.ktor:ktor-client-ios:$ktor_version")
            }
        }
        iosTest {
        }
    }
}

this is my kotlin code

class LoginApi {
    private val client = HttpClient()

    fun login(param: LoginReq, callback: (String) -> Unit) {
        GlobalScope.apply {
            launch(ApplicationDispatcher) {
                val result: String = client.post {
                    url("http://10.76.34.177:3000/user/login")
                    parameter("userName", param.userName)
                    parameter("passWord", param.passWord)
                }

                callback(result)
            }
        }
    }
}

I'm very newly in kotlin, So I will very appreciate of you reply.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/874

Ktor Version

1.1.1

Ktor Engine Used(client or server and name)

N/A

JVM Version, Operating System and Relevant Context

N/A

Feedback

When modeling a large API using a ktor HTTP client it would very useful to allow resolving a string against another URL.

Usually one would have a base URL that includes the common things (protocol, host, base path etc) and each endpoint only knows specific things. For example this would be nice:

val baseUrl = Url("https://example.com/api/v1/")
val client = HttpClient(...)
suspend fun getStuff() =
    // Would get stuff from https://example.com/api/v1/stuff
    client.get<Stuff>(baseUrl.resolve("stuff"))

Here are some more examples of how it would work:

// Sibling when the base URL doesn't end with a /
Url("https://example.com/api/v1").resolve("foo") // => https://example.com/api/foo
// Child when the base URL doesn't end with a /
Url("https://example.com/api/v1/").resolve("foo") // => https://example.com/api/v1/foo
// Root when the appending URL starts with a /
Url("https://example.com/api/v1/").resolve("/foo") // => https://example.com/foo
// Replacing when the appending URL is a full URL
Url("https://example.com/api/v1/").resolve("https://other.com") // => https://other.com
// Protocol inferred from other URL
Url("https://example.com/api/v1/").resolve("//other.com") // => https://other.com

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1137

Ktor application can't be compiled with JDK >=9. I'm getting a lot of such errors:

error: module kotlinx.io.jvm reads package io.ktor.http from both ktor.server.core and ktor.http.jvm
error: module kotlinx.io.jvm reads package io.ktor.http.content from both ktor.server.core and ktor.http.jvm
error: module kotlinx.io.jvm reads package io.ktor.util from both ktor.server.core and ktor.utils.jvm
error: module kotlinx.coroutines.jdk8 reads package io.ktor.http from both ktor.server.core and ktor.http.jvm
error: module kotlinx.coroutines.jdk8 reads package io.ktor.http.content from both ktor.server.core and ktor.http.jvm
error: module kotlinx.coroutines.jdk8 reads package io.ktor.util from both ktor.server.core and ktor.utils.jvm
error: module kotlinx.coroutines.core reads package io.ktor.http from both ktor.server.core and ktor.http.jvm
error: module kotlinx.coroutines.core reads package io.ktor.http.content from both ktor.server.core and ktor.http.jvm
error: module kotlinx.coroutines.core reads package io.ktor.util from both ktor.server.core and ktor.utils.jvm
...................

Support of Java 8 ends quite soon, so it makes sense to start supporting Jigsaw. IMHO, this should be prioritized, as it will require breaking backward compatibility. The less Ktor is popular, the easier would be to break backward compatibility. And Ktor will definitely become more popular over time.

Ktor Version

1.2.0

JVM Version, Operating System and Relevant Context

JDK 12

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1863

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2

Describe the bug
Request must include Host parameter(see https://tools.ietf.org/html/rfc2616#section-14.23)

To Reproduce
Steps to reproduce the behavior:
Run code:

import io.ktor.http.cio.parseRequest
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking

fun main() {
    test(
        """
        GET / HTTP/1.1
        

        """.trimIndent()
    )
}

fun test(http: String) {
    val request = runBlocking { parseRequest(ByteReadChannel(http)) }!!
    println("method=${request.method};uri={${request.uri}};headers=[${request.headers}]")
}

Expected behavior
An error expected - exception or null result.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1862

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2

Describe the bug
"/", "?", "#", "@" symbols in 'Host' parameter value are prohibited(see https://tools.ietf.org/html/rfc2396#appendix-A)

To Reproduce
Steps to reproduce the behavior:
Run code:

import io.ktor.http.cio.parseRequest
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking

fun main() {
    listOf(
        """
        POST / HTTP/1.1
        Host: www/example.com


        """.trimIndent(),
        """
        POST / HTTP/1.1
        Host: www.example?com


        """.trimIndent(),
        """
        POST / HTTP/1.1
        Host: www.ex#mple.com


        """.trimIndent(),
        """
        POST / HTTP/1.1
        Host: www.ex@mple.com


        """.trimIndent()
    ).forEach {
        test(it)
    }
}

fun test(http: String) {
    val request = runBlocking { parseRequest(ByteReadChannel(http)) }!!
    println("method=${request.method};uri={${request.uri}};headers=[${request.headers}]")
}

Expected behavior
An error expected - exception or null result.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1833

Subsystem
Server

Is your feature request related to a problem? Please describe.
There is currently no way to build a Parameters instance from a Map, even though the internal representation of the values is a map. Parameters can only be constructed using:

• parametersOf()
• parametersOf(name: String, value: String)
• parametersOf(name: String, values: List<String>)
• parametersOf(vararg pairs: Pair<String, List<String>>)

ParametersImpl is internal and cannot be constructed directly.

Describe the solution you'd like
I'd like there to be a parametersOf(parameters: Map<String, List<String>>):

fun parametersOf(vararg parameters: Map<String, List<String>>): Parameters =
    ParametersImpl(parameters)

Motivation to include to ktor
Seems like it should be there. There are many cases where you're already working with a parameters map, and having to transform that into a list of pairs just to have it retransformed back under the hood seems wasteful.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1221

Ktor Version: 1.2.2

Ktor Engine Used: Client - OkHttp, followRedirects set to false

Operating System: JVM 1.8 Windows 10 64-bit

Issue:
A site I'm trying to access redirects me to a path similar to "http://<url>/<page>?PartOf?Name=Value"
Ktor will encode this to "http://<url>/<page>?PartOf%3Name=Value"
According to https://tools.ietf.org/html/rfc3986#section-3.4 this is incorrect, as any ? following the first should be treated the same as a normal character.
If redirects are handled by the engine itself (e.g. OkHttp) this is handled correctly. In my case i need access to redirects though so this is not an option

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1186

In case of any errors, either a faulty route which throws an exception or requests to a non-existing route, the interceptor chain behaves differently than on requests to routes which responds properly.

In a normal situation the ApplicationSendPipeline is executed during the execution of the ApplicationCallPipeline, which enables us to trace an entire request/response situation using e.g. the Monitoring phase:

intercept(ApplicationCallPipeline.Monitoring) {
     try {
         log.info("incoming request ${call.request.httpMethod.value} ${call.request.uri}")
         proceed()
     } catch (err: Throwable) {
         log.info(err.message, err)
         throw err
     } finally {
         log.info("responding with ${call.response.status()}")
     }
}

Because we are executing within the ApplicationCallPipeline we get the added bonus that our log messages gets enriched with stuff like CallId.

However, when an error occurs (because the route does not exist, or the handler threw an exception) the ApplicationCallPipeline gets interrupted. Then some error handling is done by the DefaultEnginePipeline which tries to set a reasonable status code (404 in case of a non-existing route, 500 in case of exceptions) and logs the exception (no CallId here!).
But because the ApplicationCallPipeline got interrupted, the Monitoring interceptor from above has already exited, so it did never get a chance to act on the response code set by DefaultEnginePipeline.

After DefaultEnginePipeline has done its thing, the ApplicationSendPipeline is executed with the new response. And because ApplicationSendPipeline does not share any attributes with the ApplicationCallPipeline, we do not get the bonus of having things like CallId.

In short, it's hard to properly log a request with the resulting status code, with things like CallId. Because if we want to handle the quirks I've just mentioned, we have to do something like:

intercept(ApplicationCallPipeline.Monitoring) {
     try {
         log.info("incoming request ${call.request.httpMethod.value} ${call.request.uri}")
         proceed()
     } catch (err: Throwable) {
         log.info(err.message, err)
         throw err
     }
}

// ensure that the callId is present during the whole ApplicationSendPipeline execution
sendPipeline.intercept(ApplicationSendPipeline.Before) {
    try {
        MDC.put("call_id", call.callId)
        proceed()
    } finally {
        MDC.remove("call_id")
    }
}

sendPipeline.intercept(ApplicationSendPipeline.After) { message ->
    (when (message) {
        is OutgoingContent -> message.status
        is HttpStatusCode -> message
        else -> null
    } ?: context.response.status())?.let { status ->
        log.info("responding with ${status.value}")
    } ?: log.info("sending unknown response code")
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/833

Hi, I'm using io.ktor:ktor-network-tls:1.0.1[1], running JDK8 on MacOSX 10.10.5
The minimal working example to reproduce the problem is as follows:

import io.ktor.network.selector.ActorSelectorManager
import io.ktor.network.sockets.aSocket
import io.ktor.network.sockets.isClosed
import io.ktor.network.sockets.openReadChannel
import io.ktor.network.sockets.openWriteChannel
import kotlinx.coroutines.Dispatchers.Unconfined
import kotlinx.coroutines.newSingleThreadContext
import kotlinx.coroutines.runBlocking

/** Created by revin(caoyunbin@baidu.com) on 2018-12-26. */
fun main(args:Array<String>)=runBlocking(Unconfined){
    val tcp=aSocket(ActorSelectorManager(newSingleThreadContext("httpIO"))).tcp()
    val socket=tcp.connect("www.baidu.com",80)
    socket.openReadChannel();socket.openWriteChannel()
    socket.close();println(socket.isClosed)
}

The result expected from the above code is true but instead I got false

1. I can't use io.ktor:ktor-network-tls:1.1.0, which is the latest version as of writing. The following error shows up when trying to build the project. So I'm using an earlier version to report the problem

CONFIGURE FAILED in 7s
Could not find org.jetbrains.kotlinx:atomicfu:0.12.0.
Searched in the following locations:
    https://repo1.maven.org/maven2/org/jetbrains/kotlinx/atomicfu/0.12.0/atomicfu-0.12.0.pom
    https://repo1.maven.org/maven2/org/jetbrains/kotlinx/atomicfu/0.12.0/atomicfu-0.12.0.jar
Required by:
    project :jvm > io.ktor:ktor-network-tls:1.1.0
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > org.jetbrains.kotlinx:kotlinx-io-jvm:0.1.2
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > org.jetbrains.kotlinx:kotlinx-coroutines-io-jvm:0.1.2
    project :jvm > io.ktor:ktor-network-tls:1.1.0 > io.ktor:ktor-network:1.1.0

2. According to maven(https://mvnrepository.com/artifact/io.ktor/ktor-network-tls/1.1.0), ktor-network-tls relies on ktor-http-cio-native, is that intentional?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1686

Ktor Version and Engine Used (client or server and name)
ktor-network 1.3.1

Describe the bug
Using the slightly modified Socket Echo Server sample code from the ktor docs:

fun main() {
    runBlocking {
        val server = aSocket(ActorSelectorManager(Dispatchers.IO)).tcp().bind(InetSocketAddress("127.0.0.1", 2323))
        println("Started echo telnet server at ${server.localAddress}")

        while (true) {
            val socket = server.accept()

            launch {
                println("Socket accepted: ${socket.remoteAddress}")

                val input = socket.openReadChannel()
                val output = socket.openWriteChannel(autoFlush = true)

                try {
                    while (true) {
                        val line = input.readUTF8Line()

                        println("${socket.remoteAddress}: $line")
                        output.write("$line\r\n")
                    }
                } catch (e: Exception) {
                    // don't print stack trace so we can differentiate
                    //e.printStacktrace()
                    println("caught exception: ${e.message}")
                    socket.dispose()
                }
            }
        }
    }
}

If a client connects and at some point the connection is closed by the client, an IOException is thrown, and caught, but the stacktrace is still printed in the console. It's important to note that the echo server continues running and still accepts new connections.

This does not seem to be a coroutines issue since the following code catches the exception, but no extra stack trace is printed in the console:

fun main() {
    runBlocking {
        launch {
            try {
                throw IOException("test")
            } catch (e: Exception) {
                println("caught exception: ${e.message}")
            }
        }
    }
}

To Reproduce
Using the above server code:

1. start the server
2. connect to it (eg telnet localhost 2323)
3. close the connection (eg close the terminal window)

Expected behavior
Since the exception is caught, it is expected that the stack trace is not printed, since printing the stack trace is not configured anywhere explicitly.

Screenshots
Echo server output:

Coroutine only output:

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/742

Blocking coroutines can cause dead-locks with async filtering/processing coroutines (such as compression) on single-threaded dispatcher (single-threaded or pinned to thread event loop).

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/541

Simple demo:

https://github.com/ktorio/ktor-samples/compare/master...marshallpierce:html-builder-status-pages-swallows-exceptions?expand=1

Expected behavior: I would expect the default to involve a 500 and the exception being logged somewhere, hopefully customizable via StatusPages. Rendering part of the DSL's execution with a 200 is highly misleading, to say the least.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/429

https://github.com/ktorio/ktor/blob/7092c30706f34242419a5f0c81445dca06461dc3/ktor-http/src/io/ktor/http/URLUtils.kt#L28

should the query string from URI be considered encoded already and skip the decode and re-encode process?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/893

When configuring a very very long session duration it may cause integer overflow. It need to be decided what should be done in this case:

• fail at attempt to set duration
• bump max-age to Int.MAX_VALUE or Long.MAX_VALUE (long type requires changes, probably breaking changes)

This is related to #892.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/525

It would be really great if when using: respondHtml, styleLink, script and img tags are "auto" pushed just like JSF does: https://www.ibm.com/developerworks/library/j-javaee8-servlet4/index.html

This way there is no need for this:

get("/"){   
   call.push("some.css")
   ...
   call.respondHtml { head { styleLink("some.css") ...
   }
}

Of course there has to be some logic whether they are relative/absolute and it might require some collaboration from kotlinx.html

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/411

I've noticed that in this test https://www.techempower.com/benchmarks/#section=data-r15&hw=ph&test=plaintext ktor result are pretty bad compared to Rapidoid, Vertx, Netty.

Here's source code of this benchmark:
https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Kotlin/ktor/src/main/kotlin/org/jetbrains/ktor/benchmarks/Hello.kt

Is Ktor really that slow or there is some problem in benchmark?

That benchmark looks better for Ktor:
https://github.com/orangy/http-benchmarks

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/268

https://ktor.io/clients/http-client/features/timeout.html

The Client Timeout Feature states that "By default all these timeouts are infinite"; this doesn't seem to be the case with either the Apache or CIO engines where unless I explicitly set a timeout I get:

Apache

io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=http://localhost:1080/api/import/tickets/v1/file, socket_timeout=unknown] ms
	at io.ktor.client.features.HttpTimeoutKt.SocketTimeoutException(HttpTimeout.kt:170)
	at io.ktor.client.engine.apache.ApacheHttpRequestKt$sendRequest$$inlined$suspendCancellableCoroutine$lambda$2.failed(ApacheHttpRequest.kt:44)
	at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
	at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.SocketTimeoutException: 10,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
	... 11 common frames omitted

CIO (which doesn't seem to accept specifying timeouts at all)

kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 15000 ms
	at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:149)
	at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:119)
	at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:493)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:272)
	at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
	at java.base/java.lang.Thread.run(Thread.java:834)

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/283

This page has CIO sockets sample but has no explicit ActorSelectorManager close.

https://ktor.io/servers/raw-sockets.html#simple-client-connecting-to-an-echo-server

I came across Ktor looking for a HTTP client that could do streaming but in specific chunks. I found it to be insanely difficult to do using Netty, Spring WebClient, Reactor HttpClient etc. I was able to do it in Ktor comparatively easily but came across few issues:

1. Streaming is mentioned in the document almost as a joke, not to mention the shown code doesn't compile. https://ktor.io/clients/http-client/quick-start/streaming.html
2. There's no mention of the client Maven GAVs, anywhere, in the docs. After some digging around, I came across engines, and assumed that those correspond directly to the client libraries. This can be much clearly documented.
3. The API seems to be too keen on throwing exceptions. For instance, reading n bytes from a stream fails with an exception when the last chunk is < n bytes. Really?
4. I ended up with the following code, but it seems too much boilerplate that the library could easily do.

return flow {
    HttpClient(CIO).use {
        it.get<HttpStatement>(uri.scheme, uri.host, uri.port, uri.path).execute { response: HttpResponse ->
            val channel = response.receive<ByteReadChannel>()
            var bytesRead = 1
            while (bytesRead > 0) {
                val buffer = ByteBuffer.allocate(chunkSize)
                val packet = channel.readRemaining(buffer.capacity().toLong(), 1)
                bytesRead = packet.readAvailable(buffer)
                packet.release()
                if (bytesRead > 0) emit(buffer.flip())
            }
        }
    }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/902

I read this tutorial https://ktor.io/advanced/utilities.html. This states that parseUrlEncodedParameters(...) is used to parse the form-url encoded data from the request body, but then I looked at the function source, the comment said Parse URL query parameters. Shouldn't be used for urlencoded forms because of "+" character. I assume this function is only ment to be used on the actual URL parameter, but not on the body.
I also found a function parseQueryString(...) which produces the correct result on the body.
Is this just an oversight in the docs, or am I missing something?

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/203

Looking at ktor as an alternative to spring boot and wondering if I can migrate my SOAP Service s built by Spring Boot to ktor?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1089

Ktor Version

master

Feedback

Currently, the configuration defaults for the Session CookieConfiguration defaults to an insecure configuration.

The power of defaults cannot be overstated in a security context.

Instead of defaulting to being insecure, the session cookie should instead default to secure.

Requiring the user to opt-out of the security of these Cookie protections means far fewer default servers will be vulnerable to HTTP downgrade attacks exposing session cookies and will prevent XXS attacks from being leveraged to steal sessions.

https://github.com/ktorio/ktor/blob/583760a04d07c985376ad2f19e656d365adb9a0f/ktor-server/ktor-server-core/jvm/src/io/ktor/sessions/SessionTransportCookie.kt#L83-L86

https://github.com/ktorio/ktor/blob/583760a04d07c985376ad2f19e656d365adb9a0f/ktor-server/ktor-server-core/jvm/src/io/ktor/sessions/SessionTransportCookie.kt#L88-L91

This is not technically a "security vulnerability", it's just an insecure default. This is not unique to Ktor, many libraries default to this "insecure by default" model, but there's no reason why Ktor needs to help perpetuate the insecurity.

I might classify this under CWE-276: Incorrect Default Permissions.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/236

https://ktor.io/docs/testing.html#preserving-cookies

The docs mention: This method defines a session context that will hold cookies, and exposes a CookieTrackerTestApplicationEngine.handleRequest extension method to perform requests in that context..
However there is no CookieTrackerTestApplicationEngine and cookiesSession takes a lambda without parameters.

Further down it mentions: cookiesSession is not included in Ktor itself, but you can add this boilerplate to use it, but it is already included.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/293

The documentation states that "Unfortunately, the JDK’s TLS implementation doesn’t have support for ALPN, so your application engine must be configured properly."

However, Oracles documentation doesn't state anything of that:

Java SE 11 Security Developer’s Guide

Is the whole HTTP/2 chapter obsolete?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/211

I've the below code, route "/" and "/bye" are working fine, but route "login" given blank page once!!

package blog

import kotlinx.html.*
import kotlinx.html.stream.*    // for createHTML
import org.jetbrains.ktor.application.*
import org.jetbrains.ktor.auth.*
import org.jetbrains.ktor.features.*
import org.jetbrains.ktor.http.*
import org.jetbrains.ktor.response.*
import org.jetbrains.ktor.routing.*

import org.jetbrains.ktor.request.*   // for request.uri

import org.jetbrains.ktor.html.*
import org.jetbrains.ktor.pipeline.*

import org.jetbrains.ktor.host.*   // for embededServer
import org.jetbrains.ktor.netty.*  // for Netty

fun main(args: Array<String>) {
    embeddedServer(Netty, 8080, watchPaths = listOf("BlogAppKt"), module = Application::module).start()
}

fun Application.module() {
    install(DefaultHeaders)
    install(CallLogging)

    intercept(ApplicationCallPipeline.Call) { 
        if (call.request.uri == "/hi")
            call.respondText("Test String")
    }

    install(Routing) {
        get("/") {
            call.respondText("""Hello, world!<br><a href="/bye">Say bye?</a>""", ContentType.Text.Html)
        }
        get("/bye") {
            call.respondText("""Good bye! <br><a href="/login">Login?</a> """, ContentType.Text.Html)
        }
        route("/login") {
            authentication {
                formAuthentication { up: UserPasswordCredential ->
                    when {
                        up.password == "ppp" -> UserIdPrincipal(up.name)
                        else -> null
                    }
                }
            }
         
            handle {
                val principal = call.authentication.principal<UserIdPrincipal>()
                if (principal != null) {
                    call.respondText("Hello, ${principal.name}")
                } else {
                        val html = createHTML().html {
                        body {
                            form(action = "/login", encType = FormEncType.applicationXWwwFormUrlEncoded, method = FormMethod.post) {
                                p {
                                    +"user:"
                                    textInput(name = "user") {
                                        value = principal?.name ?: ""
                                    }
                                }

                                p {
                                    +"password:"
                                    passwordInput(name = "pass")
                                }

                                p {
                                    submitInput() { value = "Login" }
                                }
                            }
                        }
                    }
                    call.respondText(html, ContentType.Text.Html)
                }
            }
        }
    }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/220

Can't locate argument-less serializer for class kotlin.collections.List. For generic classes, such as lists, please provide serializer explicitly.

is the error message I'm trying to overcome here. The code that triggers it is client.get<List<Release>>(...). Yes, I know it says to provide a serializer, but I don't know how to do that and it doesn't explain how.

I've tried this:

val client = HttpClient(Apache) {
    install(JsonFeature) {
        serializer = KotlinxSerializer()
    }
}
val releases = client.get<List<Release>>(...)

and

val client = HttpClient(Apache) {
    install(JsonFeature) {
        serializer = KotlinxSerializer().apply {
            setListMapper(Release::class, Release.serializer())
        }
    }
}
val releases = client.get<List<Release>>(...)

I've also tried a few permutations along the lines of

setMapper(List::class, ArrayListSerializer(???))

but nothing that makes sense or compiles, as you can't say List<Foo>::class for the left, and you don't want to specify a specific kind of ArrayList on the right. Also the suggestion from runtime_usage.md that we use internal classes in our source code is a little ridiculous.

Actually, I just found this StackOverflow post. Is this the best way to solve this?

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/173

The getDigestFunction method with constant salt is no more supported after ktor 1.2.0.
So, below code does not work because of compile error.

val userTable = UserHashedTableAuth(getDigestFunction("SHA-256", salt = "ktor"), mapOf(
    "test" to decodeBase64("VltM4nfheqcJSyH887H+4NEOm2tDuKCl83p5axYXlF0=") // sha256 for "test"
))

https://ktor.io/servers/features/authentication/basic.html#strategy-validating-using-userhashedtableauth

This breaking change is implemented by below issue.
https://github.com/ktorio/ktor/issues/1061

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/170

In version 1.1.5 I had access to the method and url properties of my request inside my mock client, but once it was upgraded to 1.2.0 those properties were made unavailable without the @InternalAPI tag. This feels like a hack and is there a way to get those properties back?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/47

Hello, I would like an example using cookies with accept all cookies using JS client, i'm doing request to vertx server with session cookie and I can't see that cookie in the browser, where are cookies being stored?
this is my initial client setup:

HttpClient()
            {
                install(JsonFeature) {
                    serializer = KotlinxSerializer(Json.nonstrict)
                }
                install(HttpCookies) {
                    // Will keep an in-memory map with all the cookies from previous requests.
                    storage = AcceptAllCookiesStorage()


                }

                
            }

thanks in advance

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/64

I recently required the flexible environment on Google App Engine for a Ktor app, so I got started with a hello world example.

I thought it would be helpful to put a flag here for others who have this need. Possibly someone could also instruct me on where to put it in this repository, but this might not be applicable (I've only worked with Ktor for a couple of days, so I'm not sure how things work around here)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1950

Ktor Version and Engine Used (client or server and name)
https://ktor.io/quickstart/quickstart/gradle.html#intellij-improve-the-app-with-the-application-object

Describe the bug
Ktor is currently at version 1.3.2, but in the quickstart docs it is mentioned that is not yet 1.0. Referenced line is: "Since Ktor is not yet 1.0, we have custom Maven repositories for distributing our early preview artifacts."

Expected behavior
Line should be corrected.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/225

Generating files using Ktor 1.2.4 and Websocket HTTP Client Support dependency makes it so the project will not build.
build.gradle file specifies compile "io.ktor:ktor-client-websocket:$ktor_version" which will not work since ktor-client-websocket does not have v1.2.4 in the repositories.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/185

https://ktor.io/clients/http-client/features/logging.html clearly mentions io.ktor:ktor-client-logging artifact, but after adding that artifact and copy-pasting the code, it still doesn't compile because it also needs io.ktor:ktor-client-logging-jvm for example.

After adding that, it compiles, but then I need the hidden knowledge of https://ktor.io/servers/logging.html#providers (which is Server docs) to add something like implementation("org.slf4j:slf4j-simple:1.7.26")

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/255

https://github.com/ktorio/ktorio.github.io/blob/master/servers/self-signed-certificate.md

The documentation reads

To create a self-signed certificate using Ktor, you have to call the generateCertificate function.

io.ktor.network.tls.certificates.generateCertificate(File("mycert.jks"))

With

    implementation "io.ktor:ktor-network-tls:$ktor_version"

where ktor_version is 1.3.0, there is no generateCertificate function defined in io.ktor.network.tls.certificates

import io.ktor.network.tls.certificates.generateCertificate is an invalid import.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1208

Ktor Version

1.2.2

Feedback

I'm using Jackson mapping in the Client (here: https://ktor.io/clients/http-client/features/json-feature.html), but there seems to be no way to configure the mapper. Normally, I would just include a custom-configured object mapper (the same way I do for server component), but here it's just a single-constructor class tha explicitly calls for the default mapper inside.

Can we get a configuration param for the object mapper? Alternatively, a builder to specify it.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/272

I tried the kotlinx serialization example from the docs: https://ktor.io/servers/features/content-negotiation/serialization-converter.html

Upon using the example I get a deprecation warning for the „serialization(...“ Block because it has been replaced by json.
Second the example does not work. In contrast, the gson example works perfectly fine.
Either fix or remove the section, so other people don’t waste their time trying to get kotlinx.serializtion to work.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/224

The documentation for Micrometer metrics hint towards a Prometheus Registry being available for use here - https://github.com/ktorio/ktorio.github.io/blob/3e1f9c9c52ebbed7f429a05aa1c2b41ed69c6e93/servers/features/metrics-micrometer.md#L97

However, it appears that https://github.com/ktorio/ktor/pull/1105 has stalled out. Can you provide advise on if this feature will still be implemented, if the documentation needs to be updated, or something different. I would also like to know how I can help push this imitative along.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1842

Subsystem
Server - metrics

Is your feature request related to a problem? Please describe.
https://ktor.io/servers/features/metrics-micrometer.html

• I'm new to ktor and trying to have prometheus metrics, I do it in java app servers with simpleclient_servlet very easy
• I read install MicrometerMetrics, PrometheusMeterRegistry -> in doc missing dependency
• There is no doc where the metrics are exposed(URI)
• There is no doc if metrics need auth and which type of auth
• There is no doc should I need a route which expose metrics

Describe the solution you'd like

• More clear docs or working sample

Motivation to include to ktor
Monitoring of app is very important and prometheus is very popular

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/161

Hi! I'd like to migrate full site to hugo, so it will build faster and much simpler for website authors (since hugo distributing as single binary).

What your opinion on this?

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/105

Problem

As a new Ktor user I wanted to read the entire manual after getting started with the "Quick Start" section and before doing any serious coding.

It got frustrating reading the "Servers" section because once I'd reached the end of a long page, I'd have to scroll back towards the top, find the active navigation link, and then click on the next one.

Solution

Add previous and next links at the bottom of each page that has a category. I have a local branch that implements this approach that I could merge and use to submit a PR after I finish it.

Update

I have a working branch here: https://github.com/kensiprell/ktorio.github.io/tree/prev-next. I don't want to create a PR until someone takes a look because it touches so much.

_layouts/page.html:

        {% if page.status == 'todo' %}
          Coming soon...
        {% else %}
          {{ content }}
        {% endif %}

        {% if page.category != null %}
            {% include prev-next.html %}
        {% endif %}
      </div>
    </div>
  </div>
</div>

_includes/prev-next.html:

{% assign entries = site.pages | sort: "path" %}
{% for entry in entries %}
    {% if entry.url == page.url %}
        {% assign prevIndex = forloop.index0 | minus: 1 %}
        {% assign nextIndex = forloop.index0 | plus: 1 %}
        <div class="container-fluid col-sm-12" style="padding-top:2rem;border-top:1px solid #ddd;">
            <div class="row">
                <div class="col-sm-6">
                    {% if prevIndex %}
                        {% if entries[prevIndex].category == page.category %}
                            <a href="{{ entries[prevIndex].url }}">
                                &larr; {{ entries[prevIndex].title }}
                            </a>
                        {% else %}
                            First page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
                <div class="col-sm-6 text-right">
                    {% if nextIndex %}
                        {% if entries[nextIndex].category == page.category %}
                            <a href="{{ entries[nextIndex].url }}">
                                {{ entries[nextIndex].title }} &rarr;
                            </a>
                        {% else %}
                            Last page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
            </div>
        </div>
    {% endif %}
{% endfor %}

Here are some images showing the proof of concept.

First page in the "Kotlinx" category:

Second page in the "Kotlinx" category:

Last page in the "Kotlinx" category:

The real work involves making children directory names match the file names and then adding permalinks (see image below). I used Jekyll for the first time two days ago, so I'm not 100% certain that my approach won't have negative ramifications elsewhere. However, I haven't noticed any issues running it locally on my computer.

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/197

/master/quickstart/guides/3chat.md#L176

The exercise challenges the reader to send and receive "VOs" using web sockets. There isn't much context as to what a "VOs" is/are. Is it some kind of abbreviation? What is a "VOs"?

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/160

The text for configuring engines reads:

Every engine config has two common properties that can be set:

The dispatcher property is the CoroutineDispatcher used when processing client requests.
The sslContext is a javax.net.ssl.SSLContext allowing you to set custom keys, a trust manager or custom source for secure random data.

But when pulling a client that's non-jvm specific (like the ktor-client-cio package), I noticed that you can't configure sslContext for it, but it would make sense that that's the case since sslContext refers to javax.net.ssl.SSLContext. Should the text be changed to reflect that?

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/177

During ktorio/ktor#1133 investigation I found that currently maven examples are misleading for ktor client features.
Suggested snipped advises to plug in common module instead of JVM one (from this page):

<project>
    ...
    <dependencies>
        <dependency>
            <groupId>io.ktor</groupId>
            <artifactId>ktor-client-auth</artifactId>
            <version>${ktor.version}</version>
            <scope>compile</scope>
        </dependency>
    </dependencies>
</project>

Expected output:

<project>
    ...
    <dependencies>
        <dependency>
            <groupId>io.ktor</groupId>
            <artifactId>ktor-client-auth-jvm</artifactId>
            <version>${ktor.version}</version>
            <scope>compile</scope>
        </dependency>
    </dependencies>
</project>

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/195

master/quickstart/quickstart/gradle.md#L235

A note on the gradle quickstart page advises the reader to see the gradle quickstart page...

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/96

Pages like https://ktor.io/quickstart/quickstart/gradle.html lack ready-to-download (and use) example projects

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1004

Ktor Version

1.1.3 with Tomcat and Gradle

Operating System

Operating system: MacOS High Sierra

Feedback

By following the steps in documentation to generate a FatJar, whenever I try to execute my Ktor application by using the command java -jar pathToFatJar, I receive de following error:

Exception in thread "main" java.lang.UnsupportedOperationException: Packages and file facades are not yet supported in Kotlin reflection. Meanwhile please use Java reflection to inspect this class: class com.tidelevel.ApplicationKt
	at kotlin.reflect.jvm.internal.KClassImpl.reportUnresolvedClass(KClassImpl.kt:301)
	at kotlin.reflect.jvm.internal.KClassImpl.access$reportUnresolvedClass(KClassImpl.kt:43)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$descriptor$2.invoke(KClassImpl.kt:53)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$descriptor$2.invoke(KClassImpl.kt:44)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:92)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:31)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getDescriptor(KClassImpl.kt)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$objectInstance$2.invoke(KClassImpl.kt:106)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazyVal.invoke(ReflectProperties.java:62)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:31)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getObjectInstance(KClassImpl.kt)
	at kotlin.reflect.jvm.internal.KClassImpl.getObjectInstance(KClassImpl.kt:239)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createModuleContainer(ApplicationEngineEnvironmentReloading.kt:328)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:317)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:275)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:127)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:247)
	at io.ktor.server.tomcat.TomcatApplicationEngine.start(TomcatApplicationEngine.kt:108)
	at io.ktor.server.tomcat.EngineMain.main(EngineMain.kt:16)

This error just happen when trying to execute the FatJar in command line, everything works fine when running in Intellij. I have seen similar issues at:

• https://github.com/ktorio/ktor/issues/386
• https://stackoverflow.com/questions/54282790/ktor-running-fat-jar-throws-java-lang-unsupportedoperationexceptionpackages-an

By changing the mainClassName property to correspond to my own Application class, everything seems to work fine! I am not sure if the documentation needs to be updated in this page or if I did something wrong.

Hope I receive an answer from you!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/386

Followed instruction here https://ktor.io/servers/deploy.html for fat jar deployment

When I run java -jar myjar.jar , i get main class could'nt be found in io.ktor.server.netty.DevelopmentEngine.

Project runs fine in IDEA

Any idea why?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/466

It would be useful if ktor was able to set the SameSite attribute on a cookie, including as part of the sessions support (CookieIdSessionBuilder).

https://www.owasp.org/index.php/SameSite

https://ktor.io/servers/self-signed-certificate.html says :

• This function is defined in the method io.ktor.network.tls.certificates.generateCertificate in the artifact io.ktor:ktor-network-tls:$ktor_version.

But it is wrong for 1.3.x Ktor versions (see https://github.com/ktorio/ktor/issues/1612). The workaround is to use "io.ktor:ktor-network-tls-certificates:$ktor_version" for 1.3.x.
Samples also need to be fixed

Not sure if youtrack is the correct place to report issues for the init tools project, so I reported it on GitHub too: https://github.com/ktorio/ktor-init-tools/issues/29

I get an error when generating a client from openapi:

Couldn't generate ZIP. Reason: IllegalStateException: Unsupported 'Float'

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1987

Ktor Version and Engine Used (client or server and name)
1.32, client, CIO/okhttp, websockets, only affects graal native-image on Windows 10, does not affect graalvm JIT, hotspot, etc.

Describe the bug
Client web-socket session is not started on graalvm.
io.ktor.util.random - NativePRNGNonBlocking is not found, fallback to SHA1PRNG

To Reproduce
Steps to reproduce the behavior:

val httpClient = HttpClient(CIO) {
            install(WebSockets)
        }

httpClient.webSocket(
{ url(server.toString()) }, 
{
  // this is never executed, client ws session never starts.
 // ws connected but session is not started, app hangs
 debug log: io.ktor.util.random - NativePRNGNonBlocking is not found, fallback to SHA1PRNG`
}
})

Expected behavior
The client ws session should start after ws is connected.

Screenshots
NA

Using ktor-server-test-host results in logback classic being pulled in as a test dependancy, the relevent dependancy tree is below.

+--- io.ktor:ktor-server-test-host -> 1.3.2
|    +--- io.ktor:ktor-client-tests-jvm:1.3.2
|    |    +--- ch.qos.logback:logback-classic:1.2.3
|    |    |    +--- ch.qos.logback:logback-core:1.2.3
|    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1830

Hi!

I am working on a tool for detection of violations on design principles in my master thesis (read more about it https://github.com/Mkohm/detekt-hint), and is looking for an open source project where I can integrate my tool for some feedback and testing. Would you welcome such an integration? It would mean a lot, and I hope it can continue to improve the maintainability of this project.

It only requires creating a configuration file and a new GitHub action. Please let me know if it is of any interest, and I will create a PR with the required changes.

There is nothing in io.ktor:ktor-client-cio:1.4.0 on central repo.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1942

It is in JCenter, but not in Central.

I noticed you guys use Bintray to synchronize with Maven Central. Been there, done that. My advise: don't. Instead, use both Nexus Staging plugin and Nexus Publish Plugin.

Note that both plugins are being merged into a new Nexus Publish Plugin, but that's a work in progress. In Micronaut, we still use the former 2, and other than Maven Central being down, it is a much faster and reliable process.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/378

Feel free to rename this issue, once the problem is located. Reported by @qweryty at slack:

Giving /test?error=error and this snippet, the param route is not handled:

fun main(args: Array<String>) {
    val server = embeddedServer(Netty, 9995) {
        install(DefaultHeaders)
        install(Compression)
        install(CallLogging)

        routing {
            route("test") {
                param("error") {
                    handle {
                        call.respondText { "error" } // Not handled
                    }
                }

                get {
                    call.respondText { "test" }
                }
            }
        }
    }

    server.start()
}

With the trace feature from the wip 0.9.2, this is the output:

// /, segment:0 -> SUCCESS @ /test/(method:GET))
//   /test, segment:1 -> SUCCESS @ /test/(method:GET))
//     /test/[error], segment:1 -> SUCCESS @ /test/[error])
//     /test/(method:GET), segment:1 -> SUCCESS @ /test/(method:GET))

Even when not the same, since the previous sample should match all the methods for the param, this other sample would work (matching only get requests for the error param):

routing {
    route("test") {
        method(HttpMethod.Get) {
            param("error") {
                handle {
                    call.respondText { "error" } // Matched
                }
            }
            handle {
                call.respondText { "test" }
            }
        }
    }
}
// /, segment:0 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
//   /test, segment:1 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
//     /test/(method:GET), segment:1 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
//       /test/(method:GET)/[error], segment:1 -> SUCCESS @ /test/(method:GET)/[error])

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/50

It would be nice to have some unit tests, e.g. for the sse example, which show how basic ktor mechanisms can be tested on the lowest (unit test) level.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/44

Hi guys, I am testing to create a simple API using Ktor tomcat-war sample. I am able to run it as expected but I am getting an error when I tried to copy the war file into a different tomcat.

I run
./gradlew tomcat-war:war

I copy the war file created on libs folder into my tomcat:

And then, I try to access the project on Tomcat like so:

It just does not work. Am I missing something?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/46

Here is the output:

oleg-laptop:ktor-samples oleg$ ./gradlew :jetty-embedded:run

> Configure project :client-mpp
Kotlin Multiplatform Projects are an experimental feature.

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':client-mpp'.
> SDK location not found. Define location with sdk.dir in the local.properties file or with an ANDROID_HOME environment variable.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s

In case some additional configuration is required it should be mentioned in the README.md file.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/38

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/41

MacBook-Pro-Marcin:ktor-samples marcin$ ./gradlew :fullstack-mpp:run
Starting a Gradle Daemon, 3 stopped Daemons could not be reused, use --status for details

> Configure project :client-mpp
Kotlin Multiplatform Projects are an experimental feature.

> Configure project :fullstack-mpp
Kotlin Multiplatform Projects are an experimental feature.

FAILURE: Build failed with an exception.

* Where:
Build file '/Users/marcin/Projects/ktor-samples/mpp/fullstack-mpp/build.gradle' line: 81

* What went wrong:
Could not determine the dependencies of task ':fullstack-mpp:run'.
> No such property: files for class: org.jetbrains.kotlin.gradle.plugin.mpp.DefaultKotlinCompilationOutput

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 31s
MacBook-Pro-Marcin:ktor-samples marcin$ 

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/22

This example run app in ROOT for example http://localhost:8080/
How run app in http://localhost:9090/appktor/ ?

Thanks

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/32

The documentation is fine and all, and does what it needs to do, however, there is less work for a Ktor application developer to do when using Jib.

I contributed an example myself based on various snippets and videos I could find around (for my first use of Kotlin and Ktor, I was pretty happy about it, so thanks to the community for that 👍)

Ref https://github.com/GoogleContainerTools/jib/pull/1291

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/58

Hi,

My sourceSets has "androidNativeArm32/Arm64/X86/X64"

But no one implementation can use

implementation("io.ktor:ktor-client-core:$ktor_version")
implementation("io.ktor:ktor-client-core-native:$ktor_version")
implementation("io.ktor:ktor-client-android:$ktor_version")
implementation("io.ktor:ktor-client-curl:$ktor_version")

like my androidNativeArm32Main can't import io.ktor... (like HttpClient)

which can I use

Tnank you very much

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/70

The docker deployment example (doc, source), from what I gathered, works by building on the host and copying the binary to the image being built.

On top of being a bit unusual, I also feel that it is a bit of a bad idea: the build process could be platform specific, ending up with an unsuitable jar in a linux container. I don't know java-based build processes enough to judge if this is indeed a problem or not, but it is certainly the case with composer(php) and npm(node.js).

Also, I'll be honest - I was looking for a quick way to build a docker image via gitlab and followed these steps. Unfortunately, there are some missing steps due to this situation (need to configure my ci env to use a jre+gradle image).

Edit: Just came across the following article, maybe it's good for inspiration: https://codefresh.io/docs/docs/learn-by-example/java/gradle/

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/52

JDK 12
Kotlin version 1.3.50-release-112

gradle appengineRun

FAILURE: Build failed with an exception.

• Where:
  Build file '/Users/rojepp/dev/steward/build.gradle' line: 11

• What went wrong:
  A problem occurred evaluating root project 'steward'.
  Could not get unknown property 'kotlin_version' for object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/54

Would make a great addition to this repo 😁 @cy6erGn0m

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/53

There were some exceptions when I built the project:
A problem occurred evaluating root project 'lzucxm'. Cannot add task 'run' as a task with that name already exists.
So, I'm very gratitude if anyone give me some help.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/379

Hi, i'm using ktor for simple rest server on Android app. Everything works fine, but if i run proguard, then hell is unleashed and i have over 680 issues with progurad.
100% problems is resolved by:

-dontwarn io.netty.**
-dontwarn io.ktor.**
-dontwarn com.typesafe.**

But server is not responding after start so i'm assuming, that proguard rules should be different :)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/953

CWE-829: Inclusion of Functionality from Untrusted Control Sphere

All of these build files include resolving these dependencies over HTTP instead of HTTPS. Any of these artifacts could have been MITM to maliciously compromise them and infect the build artifacts that were produced. Additionally, if any of these JAR files were compromised, any developers using these dependencies now could continue to be infected.

• https://github.com/ktorio/ktor/blob/0.9.5/build.gradle
• https://github.com/ktorio/ktor/blob/0.9.4/build.gradle
• https://github.com/ktorio/ktor/blob/0.9.3/build.gradle
• https://github.com/ktorio/ktor/blob/0.9.2/build.gradle
• https://github.com/ktorio/ktor/blob/0.9.1/build.gradle
• https://github.com/ktorio/ktor/blob/1.0.0/build.gradle
• https://github.com/ktorio/ktor/blob/1.0.1/build.gradle

This vulnerability has a CVSS v3.0 Base Score of 8.1/10
https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H

This isn't just theoretical; POC code exists already to maliciously compromise jar file inflight.

See:

• https://max.computer/blog/how-to-take-over-the-computer-of-any-java-or-clojure-or-scala-developer/
• https://github.com/mveytsman/dilettante

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/74

Especially the MPP sample. Please don't make developer's lives more difficult by letting broken code rot here on GitHub.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/19

Not only are the example outdated (code is deprecated), but its also not working, thus causing quite a lot of confusion; (especially since the documentation regarding the Form authorization is pretty limited);

I can create a PR for a suitable minor change to make it work if you want :)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/37

I've tried to build fullstack-mpp project by opening build.gradle in IntelliJ Idea and then adding gradle.properties file with following content:

ktor_version=1.1.1
kotlin.code.style=official
kotlin_version=1.3.11
logback_version=1.2.1

When trying to run it, I get following message:

FAILURE: Build failed with an exception.

* Where:
Build file 'C:\Users\czare\IdeaProjects\fullstack-mpp\build.gradle' line: 81

* What went wrong:
Could not determine the dependencies of task ':run'.
> No such property: files for class: org.jetbrains.kotlin.gradle.plugin.mpp.DefaultKotlinCompilationOutput

I didn't change anything in build.gradle file. What am I doing wrong?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/35

Using commit 5480a4d8eb338302f181c054b2a067e67ff7e366

1. git clone
2. ./gradlew build
3. Error:

Could not resolve all files for configuration ':client-mpp:iosArm64CompileKlibraries'.

Could not find io.ktor:ktor-client-ios_debug_ios_arm64:1.1.1.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/349

The problem reproduces with http2-push from https://github.com/ktorio/ktor-samples

1. Run ./gradlew :http2-push:run
2. Navigate to http://localhost:8080

Resulting exception:

2018-03-07 14:44:55.948 [nettyCallPool-4-1] ERROR Application - Unhandled: GET - /
java.lang.IllegalArgumentException: ParametersBuilder can only build a single Parameters instance
        at io.ktor.http.ParametersBuilder.build(Parameters.kt:26)
        at io.ktor.http.URLBuilder.appendTo(URLBuilder.kt:46)
        at io.ktor.http.URLBuilder.buildString(URLBuilder.kt:62)
        at io.ktor.server.engine.BaseApplicationResponse.push(BaseApplicationResponse.kt:188)
        at io.ktor.routing.RoutingApplicationResponse.push(RoutingApplicationCall.kt)
        at io.ktor.http.PushKt.push(Push.kt:26)
        at io.ktor.http.PushKt.push(Push.kt:13)
        at io.ktor.http.PushKt.push(Push.kt:9)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/538

According to users, if a user has poor connection some resources and scripts received by Chrome web browsers are corrupted

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/31

Expected

I'm attempting to have my Ktor app's main method run automatically once deployed to AppEngine, similar to how a Jars main method runs when uploaded to AppEngine.

Actual

Currently the Ktor application's main method is only called when the app's hosted route is called: https://[yourProjectName].appspot.com.

In the applications's main method is logic to retrieve content from an API request based on a Timer Task and save that information to a Firestore database which a client consumes. The goal is to have this task running continuously.

This logic currently works when deployed in a Jar to AppEngine. However, implementing this with Ktor would save deploy time and help future proof the backend service if endpoints are required.

Setup

Main Method

import io.ktor.application.Application
fun Application.main() {
    // App logic here.
   Timer().scheduleAtFixedRate(_TaskName_()), 0, 60000) 
   ...
}

build.gradle

buildscript {
ext.kotlin_version = '1.3.10'
ext.ktor_version = '1.0.0'
ext.appengine_version = '1.9.60'
ext.appengine_plugin_version = '1.3.4'
ext.junitJupiterVersion  = '5.0.3'

repositories {
    jcenter()
    mavenCentral()
}
dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
    classpath "com.google.cloud.tools:appengine-gradle-plugin:$appengine_plugin_version"
    classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.3'
    }
}

plugins {
    id 'org.jetbrains.kotlin.jvm' version '1.2.51'
}

apply plugin: 'java'
apply plugin: 'kotlin'
apply plugin: 'war'
apply plugin: 'com.google.cloud.tools.appengine'

sourceSets {
    main.kotlin.srcDirs = [ 'src/main/kotlin' ]
}

sourceCompatibility = 1.8

repositories {
    jcenter()
    mavenCentral()
    maven { url "https://kotlin.bintray.com/ktor" }
}

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
    implementation "io.ktor:ktor-server-servlet:$ktor_version"
    implementation "io.ktor:ktor-html-builder:$ktor_version"
    providedCompile "com.google.appengine:appengine:$appengine_version"
    implementation 'com.squareup.retrofit2:retrofit:2.3.0'
    implementation 'com.squareup.retrofit2:converter-gson:2.3.0'
    implementation 'com.squareup.retrofit2:adapter-rxjava:2.3.0'
    implementation 'io.reactivex.rxjava2:rxjava:2.1.1'
    implementation 'com.google.firebase:firebase-admin:6.3.0'
    implementation 'com.google.apis:google-api-services-youtube:v3-rev204-1.23.0'
    testCompile group: 'junit', name: 'junit', version: '4.12'
    // JUnit Jupiter API and TestEngine implementation
    testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")
    testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")
    testCompile("org.assertj:assertj-core:3.10.0")
    // To avoid compiler warnings about @API annotations in JUnit code
    testCompileOnly('org.apiguardian:apiguardian-api:1.0.0')
}

kotlin.experimental.coroutines = 'enable'

compileKotlin {
    kotlinOptions.jvmTarget = "1.8"
}
compileTestKotlin {
    kotlinOptions.jvmTarget = "1.8"
}

task run(dependsOn: appengineRun)

appengine {
    deploy {
    version = 'media-staging-1201181257pm'
    }
}

src/main/resources/application.conf

ktor {
    application {
        modules = [ InitializationKt.main ]
    }
}

src/main/webapp/WEB-INF/

appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <threadsafe>true</threadsafe>
    <runtime>java8</runtime>
</appengine-web-app>

web.xml

<?xml version="1.0" encoding="ISO-8859-1" ?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0">
    <servlet>
        <display-name>KtorServlet</display-name>
        <servlet-name>KtorServlet</servlet-name>
    <servlet-class>io.ktor.server.servlet.ServletApplicationEngine</servlet-class>
        <!-- path to application.conf file, required -->
        <init-param>
            <param-name>io.ktor.config</param-name>
            <param-value>application.conf</param-value>
        </init-param>
    </servlet>
    <servlet-mapping>
        <servlet-name>KtorServlet</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>
</web-app>

Thanks for the help!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/543

CIO and Netty behave differently for simple hello-server (e.g. netty-embedded from samples)

embeddedServer(Netty, commandLineEnvironment(args)).start() // Works
embeddedServer(CIO, commandLineEnvironment(args)).start() // Application exits right after start call

Even though one may use start(wait = true), it would be nice for CIO to be a drop-in replacement for Netty

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/25

At the moment you cannot create hrefs from a location with trailing /:
/somepath/ as it would end up being /somepath

This should of cause also work for nested Locations, so i could do:

@location("/somepath/") class SomeLocation{
    @location("/nested/") class Nested
}

and get /somepath/and /somepath/nested/

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1743

Ktor Version and Engine Used (client or server and name)
1.32, server

Describe the bug
Sealed classes are stringified to differently when using kotlinx.serialization through Ktor content negotiation
To Reproduce
Steps to reproduce the behavior:

1. Create class with list:

@Serializable
class SealedList(val list: List<Test>)

@Serializable
sealed class Test {
    @Serializable
    class A() : Test()

    @Serializable
    class B() : Test()
}

2. Setup contentNegotiation:

fun Application.installContentNegotiation() {
    install(ContentNegotiation) {
        json(json = Json(DefaultJsonConfiguration.copy(encodeDefaults = false, prettyPrint = true)))
    }
}

3. Setup routing:

get("testing") {
    val message = SealedList(listOf(Test.A(), Test.B()))
    val json = Json(JsonConfiguration.Default.copy(prettyPrint = true))
    println(json.stringify(SealedList.serializer(), message))
    call.respond(message)
}

4. Test it

startTestServer {
    val call = handleRequest(HttpMethod.Get,"testing") {
        jsonAcceptType()
    }

    println(call.response.content)
}

Expected behavior
Json created "by hand" should be equal to this created by Ktor. Results:

1. By hand:

{
    "list": [
        {
            "type": "navigationData.server.router.Test.A"
        },
        {
            "type": "navigationData.server.router.Test.B"
        }
    ]
}

2 By ktor:

{
    "list": [
        ["navigationData.server.router.Test.A", {
            }
        ],
        ["navigationData.server.router.Test.B", {
            }
        ]
    ]
}

Maybe I didn't understand something, or Ktor needs more data, but I couldn't find it in documentation:
https://ktor.io/servers/features/content-negotiation/serialization-converter.html

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1234

Ktor Version and Engine Used (client or server and name)
version, client/server, additional ktor modules included
ktor 1.2.2
kotlin 1.3.41

dependencies

compile "ch.qos.logback:logback-classic:$logback_version"
    compile "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.9.2"
    compile "com.google.appengine:appengine:$appengine_version"
    compile "com.google.cloud:google-cloud-logging-logback:$gce_logback_version"
    compile "io.ktor:ktor-auth:$ktor_version"
    compile "io.ktor:ktor-client-apache:$ktor_version"
    compile "io.ktor:ktor-client-auth-jvm:$ktor_version"
    compile "io.ktor:ktor-client-core-jvm:$ktor_version"
    compile "io.ktor:ktor-client-core:$ktor_version"
    compile "io.ktor:ktor-html-builder:$ktor_version"
    compile "io.ktor:ktor-jackson:$ktor_version"
    compile "io.ktor:ktor-locations-metadata:$ktor_version"
    compile "io.ktor:ktor-locations:$ktor_version"
    compile "io.ktor:ktor-server-core:$ktor_version"
    compile "io.ktor:ktor-server-netty:$ktor_version"
    compile "io.ktor:ktor-server-servlet:$ktor_version"
    compile "net.logstash.logback:logstash-logback-encoder:5.1"
    compile "org.jetbrains:kotlin-css-jvm:1.0.0-pre.31-kotlin-1.2.41"
    compile 'com.google.cloud:google-cloud-firestore:1.11.0'
    compile 'com.google.cloud:google-cloud-storage:1.38.0'
    compile 'org.koin:koin-ktor:2.0.1'
    testCompile "io.ktor:ktor-server-tests:$ktor_version"
    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"

Intelij:

IntelliJ IDEA 2019.1.3 (Ultimate Edition)
Build #IU-191.7479.19, built on May 28, 2019
JRE: 1.8.0_202-release-1483-b58 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.1

Describe the bug
When installin Authentication module I need to create an Apache client. Doing that by using the constructor with engineFactory causes a weird error where Inteliji syntax is almost error like colour (#C53A47). When hovering over - no explanation, when I start the app it says that client lateinit var has never been initialized.

To Reproduce
Follow official tutorial for adding authentication.

Expected behavior
Adding http client by following factory object (Apache).

My workaround (1st line on the screenshot) is to use the constructor with engine (not engineFactory) and calling build() on Apache object.

Screenshots

Please accept my apologies if I posted this in a wrong place or this is obvious. It's just my best intention to help others to avoid this behaviour.

Thanks

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/597

Ktor has a great Locations feature, which is about type-safe routing. Imagine that one wants to get type-safe routing also in templates, e.g. to use

// somewhere in controller, once
@Location("/products")
class Products {
    @Location("/{id}")
    class Item(val id: Int)
}

// and later reuse them in every link or form
a(href=href(Products.Item(id))) { "#$id" }

instead of

// using error-prone and refactoring-hard string template to build href
a(href="/products$id") { "#$id" }

The problem is function

// converts given location into URL string
fun href(location: Any): String

definition. For now the function is available only in Routing context, and passing it explicitely into templates makes code too verbose. Like

fun Routing.productRoutes() {
    // same line per every controller function
    fun href(location: Any) : String = application.locations.href(location)

    // per every get or post call
    get<Products> { _ ->
        call.respondHtmlTemplate(
                ListTemplate(
                        title = "All products",
                        tableModel = allProductTableModel(::href),
                        href = ::href
                )
        ) {}
    }
}

instead of just

fun Routing.productRoutes() {
   get<Products> { _ ->
       call.respondHtmlTemplate(
               ListTemplate(title = "All products", tableModel = allProductTableModel())
       ) {}
   }
}

if Locations instance were available standalone, one was able to use it inside a template directly.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/539

I have a multiplatform project that makes use of common module in which my classes for API request/response are located. This is why I can't use @Location on the class itself (because common module can't have ktor jvm dependency)
I tried to create a typealias and apply @Location annotation to it but it gives compiler error because annotation doesn't target typealiases

I am not sure if this can be solved by just adding @Target(AnnotationTarget.TYPEALIAS) or needs some deeper integration but I'm sure this is a great feature to support

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1163

Ktor Version

1.2.0

Ktor Engine Used(client or server and name)

Netty Server

JVM Version, Operating System and Relevant Context

11.0.2, Debian (docker)

4 Cores -Xmx2G (parallelism = 4)

Default settings

Feedback

For most of the cases performance is pretty good, we get < 50ms including db save (all done asynchronously). Unfortunately from time to time, we are getting requests which stuck for > 1s, in worst cases even > 20 seconds. Server got the request, but processing route wasn't invoked yet. In below there is part of NewRelic instrumentation stuck for such slow call, the problem is that Ktor is not yet well instrumented, and the only consecutive calls I see taking that time are below:

0 | 0.00% | Truncated: NettyUpstreamDispatcher |   | 0.000  s
-- | -- | -- | -- | --
0 | 0.00% | HttpServerExpectContinueHandler.channelRead() | Async | 0.000  s
0 | 0.00% | HttpServerExpectContinueHandler.channelRead() | Async | 0.000  s
0 | 0.00% | RequestBodyHandler.channelRead() | Async | 0.000  s
16.0 | 0.08% | NettyApplicationCallHandler.channelRead() | Async | 20.025  s
16.0 | 0.08% | NewRelicFeature.wrapIntoNewRelicTransaction() |   | 20.025  s
16.0 | 0.08% | com.revolut.eventstore.api.write.EventsControllerKt/saveEvent |   | 20.025  s
1.0 | 0.00% | Application code (in com.*.api.write.EventsControllerKt/saveEvent)

What I am trying to understand is - what could happen between:

0 | 0.00% | RequestBodyHandler.channelRead() | Async | 0.000  s
-- | -- | -- | -- | --
16.0 | 0.08% | NettyApplicationCallHandler.channelRead() | Async | 20.025  s

Why it took such long? It's pretty hard to understand where may be any blocking/under sourced part - so I would really appreciate help with it

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1825

Ktor Version and Engine Used (client or server and name)
ktor 1.3.2 (Netty)

Describe the bug
https://api.ktor.io/1.3.2/io.ktor.request/http-method.html states:

Returns request HTTP method possibly overridden via header X-Http-Method-Override

However it doesn't work. httpMethod() returns POST while headers[" X-Http-Method-Override"] returns GET.

To Reproduce
Execute POST request with X-Http-Method-Override: GET header.

Expected behavior
Returns 'GET' or documentation is not correct.

Screenshots

Is that correct that routing API also ignores X-Http-Method-Override or it's a bug ?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1995

now just range

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1768

Subsystem
Server(/Client?):

• ktor-serialization
• ktor-websockets

Is your feature request related to a problem? Please describe.
Right now I don't see a good way to combine websockets with serialization.

Describe the solution you'd like
In http you can do it like this:

call.respond(SomeSerializableClass())

I'd propose something like this:

outgoing.send(
    Frame.Text.fromSerializable(SomeSerializableClass())
)

and

textFrame.toSerializable<SomeSerializableClass>()

Either in some kind of library or at least somewhere in the documentation.

Motivation to include to ktor
Currently there's not a really good way to communicate over websockets via JSON.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1722

Ktor Version and Engine Used (client or server and name)
ktor_version=1.3.1
server

Describe the bug
CORS feature installed in Application

   install(CORS)
    {
        method(HttpMethod.Options)
        header(HttpHeaders.XForwardedProto)
        anyHost()
        allowSameOrigin = false
        allowCredentials = true
        allowNonSimpleContentTypes = true
        maxAgeDuration = 1.toDuration(DurationUnit.DAYS)
    }

The preflight request of some browser contains Access-Control-Request-Headers, but the field-value of it is empty, function corsCheckRequestHeaders will return false, and the browser will get 403 status.

preflight request header

Host: xxx
Connection: keep-alive
Access-Control-Request-Method: POST
Origin: https://xxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36 QBCore/4.0.1295.400 QQBrowser/9.0.2524.400 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2875.116 Safari/537.36 NetType/WIFI MicroMessenger/7.0.5 WindowsWechat
Access-Control-Request-Headers:
Accept: /
Referer: https://xxx
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.5;q=0.4

corsCheckRequestHeaders()

private fun ApplicationCall.corsCheckRequestHeaders(): Boolean {

    val requestHeaders =
        request.headers.getAll(HttpHeaders.AccessControlRequestHeaders)
            ?.filter { !it.isBlank() } // add this line to fix the issue <<<<<<<<<<<<<<<<<<<<<<<
            ?.flatMap { it.split(",") }
            ?.map {
                it.trim().toLowerCasePreservingASCIIRules()
            } ?: emptyList()

    return requestHeaders.none { it !in allHeadersSet }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1447

Ktor Version and Engine Used (client or server and name)
1.2.5 and 1.3.0-beta-1, server, netty and CIO

Describe the bug
When using the base64 encoding for a session cookie, the correct cookie is not picked up.

In my repro example below I am basically calling /login, which sets the session, and then /, where the session is evaluated. When using base64 encoding, the second call doesn't succeed, as no session is found.

Important to notice that this works with all other encodings, and even with base64 in the test engine.

To Reproduce
Minimal repro here: https://github.com/AndreasVolkmann/ktor-session

Steps to reproduce the behavior:

1. Start the server
2. Go to localhost:5000/login which should return Logged in and set the session / cookie.
3. Go to localhost:5000 where an error is shown

When changing the encoding, the above scenario works.

Expected behavior
The session cookie should be set and valid for the next call, just like with all other encodings and under test.

Tested with multiple browsers and postman.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1874

Ktor Version and Engine Used 1.3.2
Additionally using ktor-gson plugin.

Describe the bug
gson converter returns a 500 status request if the gson is badly formatted. I think it should be a 400.

To Reproduce
Steps to reproduce the behavior:

1. Add content negotiation plugin. Use the gson converter
2. Call your endpoint with badly formatted json
3. Get a 500 status

Expected behavior
Should be a 400 status.

gson converter should catch the JsonSyntaxException and throw a BadRequestException

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1484

It would be great if there was a built in (ktor) feature for SinglePage Applications

there are third party ones such as

• https://github.com/lamba92/ktor-spa
• https://github.com/comchangs/ktor-easy-spa

but I would prefer an official ktor supported approach/feature

(a simple static resource is not sufficient, see #749)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/697

sorry if there is already an appropriate way of doing this what I'm going to ask for. Since the documentation is lacking with respect to the usage of the ConditionalHeaders feature I coulnd't find a way of managing it concisely.
As far as I could see, in order to utilize ConditionalHeaders the OutgoingContent's headers need to be set. Thus this here

// somewhere in installation phase
install(ContentNegotiation) {
    register(ContentType.Application.Json, GsonConverter())
}
install(ConditionalHeaders)

// somewhere in api call
ctx.call.response.header(HttpHeaders.LastModified, lastModifiedTime)
ctx.call.respond(HttpStatusCode.OK, anydata)

don't have any effect since the header of the ApplicationResponse is set (instead of the content).
Therefore when I use ContentNegotiation I would need to adapt the conversion process so that the produced OutgoingContent instances also contain the relevant headers which is imho against the concise api nature of ktor.

wouldn't it make sense that ApplicationResponse headers are applied to the OutgoingContent headers?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1744

I think this issue is related to #783

Ktor Version
1.3.1

Ktor Engine Used
server netty

JVM Version, Operating System, and Relevant Context
jdk1.8.0_201 ，Linux Mint 19

I was using gradle application plugin

running
gradle run
while on dev it was running fine

but for the case using the distribution created after
gradle build

project_dir/build/distribution when I it tried runing the application

./application_name

initial loading like creating database and redis connection instance is successful then after that the error occurred

I encountered the same issue #783

These are my dependencies:

plugins {
    java
    kotlin("jvm")
    application
}


val ktor_version = "1.3.1"
val kodein_verison = "6.5.3"
val exposedVersion = "0.22.1"
val csv_reader_version = "0.7.3"

repositories {
    mavenCentral()
    jcenter()
    maven("https://jitpack.io")
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))

    //-- local jar files
    implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))

    //ktor
    implementation("io.ktor:ktor-server-netty:$ktor_version")
    implementation("io.ktor:ktor:$ktor_version")
    implementation("io.ktor:ktor-websockets:$ktor_version")
    implementation( "io.ktor:ktor-gson:$ktor_version")

    //logging
    implementation("io.github.microutils:kotlin-logging:1.6.24")
    implementation("ch.qos.logback:logback-classic:1.2.3")

    // token verification
    implementation("io.ktor:ktor-auth-jwt:$ktor_version")

    //json handlers
    implementation("com.beust:klaxon:5.0.1")

    //encryption
    implementation("com.github.simbiose:Encryption:2.0.1")
    implementation("commons-codec:commons-codec:1.14")

    //di
    implementation("org.kodein.di","kodein-di-generic-jvm",kodein_verison)

    //redis
    implementation("io.lettuce:lettuce-core:5.2.1.RELEASE")

    //database
    implementation("org.jetbrains.exposed", "exposed-core", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-dao", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-jdbc", exposedVersion)
    implementation("org.jetbrains.exposed", "exposed-java-time", exposedVersion)

    //csv reader
    implementation("com.github.doyaaaaaken:kotlin-csv-jvm:$csv_reader_version")

    implementation("com.zaxxer:HikariCP:3.4.1")
    implementation ("mysql:mysql-connector-java:8.0.11")

    //-- jasper reports
    implementation("net.sf.jasperreports", "jasperreports", "6.4.0") {
        exclude("org.olap4j", "olap4j")
    }
    implementation("com.lowagie", "itext", "2.1.7")
}

application conf

ktor {
    enviroment = dev
    deployment {
        port = 3001
        watch = ["project_root"]
    }

    application {
        modules = [
            com.my.module1,
            com.my.module2,
        ]
    }
}

stack trace:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at kotlin.reflect.jvm.internal.calls.CallerImpl$Method.callMethod(CallerImpl.kt:97)
        at kotlin.reflect.jvm.internal.calls.CallerImpl$Method$Static.call(CallerImpl.kt:106)
        at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:106)
        at kotlin.reflect.jvm.internal.KCallableImpl.callDefaultMethod$kotlin_reflection(KCallableImpl.kt:152)
        at kotlin.reflect.jvm.internal.KCallableImpl.callBy(KCallableImpl.kt:110)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.callFunctionWithInjection(ApplicationEngineEnvironmentReloading.kt:380)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:328)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1$$special$$inlined$forEach$lambda$1.invoke(ApplicationEngineEnvironmentReloading.kt:275)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1$$special$$inlined$forEach$lambda$1.invoke(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:308)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:274)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:33)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:290)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:272)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:125)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:245)
        at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:126)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:26)
        at com.vhl.sms.caster.MainAppKt.main(MainApp.kt:47)
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method "io.ktor.application.ApplicationEvents.subscribe(Lio/ktor/application/EventDefinition;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle;" the class loader (instance of io/ktor/server/engine/OverridingClassLoader$ChildURLClassLoader) of the current class, io/ktor/websocket/WebSockets$Feature, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for the method's defining class, io/ktor/application/ApplicationEvents, have different Class objects for the type kotlinx/coroutines/DisposableHandle used in the signature
        at io.ktor.websocket.WebSockets$Feature.install(WebSockets.kt:112)
        at io.ktor.websocket.WebSockets$Feature.install(WebSockets.kt:104)
        at io.ktor.application.ApplicationFeatureKt.install(ApplicationFeature.kt:64)
        at com.vhl.sms.caster.MainAppKt$messageModule$1.invokeSuspend(MainApp.kt:105)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:271)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.vhl.sms.caster.MainAppKt.messageModule(MainApp.kt:57)
        ... 25 more

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1316

I am trying to set the Content-Length header when using call.RespondOutputStream, however I get an error "io.ktor.http.UnsafeHeaderException: Header Content-Length is controlled by the engine and cannot be set explicitly".

The issue is that there is no way for the engine to set the header itself since the stream does not provide its size.

So first of all is there a way for me to bypass this somehow in the current release and set the header myself manually?

2. Can we add the ability to set the Content-Length header which would override whatever the engine chooses in a future release?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/604

Code snippet:

fun Application.module() {
    install(CallLogging) {
        mdc("status_code") {
            it.response.status()?.toString()
        }
    }
    routing {
        get("/") {
            call.respondText("OK", contentType = ContentType.Text.Plain)
        }
    }
}

The MDC value of "status_code" is always null.

The reason is CallLogging feature evaluates all MDC values before the call gets processed.
In CallLogging.kt:

            if (feature.mdcEntries.isNotEmpty()) {
                pipeline.intercept(loggingPhase) {
                    val mdc = feature.setupMdc(call)
                    withContext(MDCSurvivalElement(mdc)) {
                        try {
                            proceed()
                            feature.logSuccess(call)
                        } finally {
                            feature.cleanupMdc()
                        }
                    }
                }
            }

Simply calling feature.setupMdc(call) after proceed() again got the issue fixed, but I'm not sure it's the correct way.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1414

Hi and thanks for this powerful library. In advance, apologies if I misunderstood some of the mechanics I describe here.

Currently, the CallLogging feature creates a "Logging" phase inserted before "Monitoring" and intercepts it to setup the MDC context for the rest of the pipeline with:

pipeline.intercept(loggingPhase) {
  withMDC {
    proceed()
    feature.logSuccess(call)
  }
}

This makes the MDC nicely available in all the sub-coroutines for reading. However any manual modification to the context in the rest of the pipeline is lost on suspension, making it effectively read-only.
From what I understand, this is expected and similar to the caveat described in the kotlinx documentation about ThreadLocal & coroutines

As a result, it very hard to add values to the MDC that are not trivially available during the Logging phase. In our use-case, these are:

• The id of the user making the request (it is computed during the authentication phase).
• The graphql operation requested by the user (it is computed in the routing phase as our graphql library parses the request).

I will try to find a hacky way to handle this for now, but it seems to me that the general use-case (of adding additional data to the MDC context later-on in the pipeline) is legitimate and it would be great to have "native" ktor support for it!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1954

Any chance to finaly add JSON RPC support ?

i see pool request

https://github.com/ktorio/ktor/pull/587

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/384

This is a follow up to #80. Apparently, responses were fixed, but there are still issues with the requests.

Request with Content-Type: application/json is decoded not as UTF-8, while request with Content-Type: application/json; charset=utf-8 is decoded correctly. Both has to behave the same and be decoded as UTF-8.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/193

Given that ktor now has Introduce send and receive pipelines on all levels(commit msg).
Wouldn't this test make sense (it fails):

   @Test
    fun testStatusMapping2() {
        withTestApplication {
            application.routing{
                route("/foo") {
                    install(StatusPages) {
                        statusFile(HttpStatusCode.NotFound, filePattern = "error#.html")
                    }
                    intercept(ApplicationCallPipeline.Call) {
                        call.respond(HttpStatusCode.NotFound)
                    }
                }
            }
            handleRequest(HttpMethod.Get, "/foo").let { call ->
                assertEquals("<html><body>error 404</body></html>", call.response.content)
            }
        }
    }

It's a copy/paste of testStatusMapping test, but with statuspages installed under a route pipeline (edited)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1305

Ktor Version and Engine Used (client or server and name)
ktor-server-core 1.2.3

Describe the bug
The XForwardedHeaderSupport feature assigns the first entry in X-Forwarded-For to call.request.origin.remoteHost. However, typically the way that proxies work is that they accept arbitrary X-Forwarded-For values from clients and add their own IPs to the end of the list. So to properly figure out a client's IP address, one needs to know how many proxies one is behind, and count that far from the end of the list.

Now, to be fair, the docs on remoteHost say to not use it for user authentication. But if you do know how many proxies you are behind, it should be safe to use it for user authentication, by counting that many IPs from the end. So it would be good if you could do this easily!

So I guess I have two separate concerns:
(a) Today's behavior is surprisingly insecure, and though it's documented on RequestConnectionPoint it should be documented on https://ktor.io/servers/features/forward-headers.html

(b) Ideally you should be able to configure XForwardedHeader with something like

install(XForwardedHeaderSupport) {
  // Given `x-forwarded-for: GARBAGE1, GARBAGE2, REALCLIENTIP1, PROXY1` and request.local being the second proxy, this will get REALCLIENTIP1
  proxyCount = 2
}

ie, "behind one proxy" means the last IP address is correct, "behind two proxies" means the second-to-last, etc.

To Reproduce

• Run with XForwardedHeaderSupport behind some proxy, send a request with X-Forwarded-For: 123.123.123.123., see request.origin.remoteHost becomes 123.123.123.123.

Expected behavior
It should be possible to configure XForwardedHeaderSupport to reliably ascertain the remoteHost.

Happy to send in a PR, esp if people like the proposed variable name.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/595

The Netty engine config allows configuring the server's SSL parameters, but it doesn't expose the SslContextBuilder . This makes it hard to create a ktor server that requires clients to supply a certificate verified against a custom CA. It would be great if the Netty engine config allowed specifying that client certificates are required, and a custom trust store for validating them.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1660

Subsystem
ktor-locations

Description
We are going to develop and expand ktor Locations (Type-Safe routing https://ktor.io/servers/features/locations.html). One of the directions to improve is Kotlin Multiplatform. The most obvious way of migration to MPP is to use kotlinx.serialization in locations. Due to the limited reflection capabilities of non-JVM targets, there are things that not so easy to implement. On the other hand, there are questionable features that may lead to issues. One of the problematic features is nested location classes and nested location objects.

What we are thinking of to change:

• a nested location class should always have a property of the outer class or object
• nested objects in objects are not allowed

The motivation for the first point is the fact that a location class nested to another, makes no sense without the ability to refer to the outer class. Consider the following example:

@Location("/api/{version}")
class Api(val version: Int) {
    @Location("/user/{id}")
    class User(val id: String, val api: Api = Api(1)) {
        // here API presence looks required, 
        // otherwise, there is no way to find a version value
        // so now we demand the api parameter to exist
    }
}

The other point is that one can't just move a nested class outside without additional manual changes:

@Location("/root")
class Outer {
    @Location("/child")
    class Child  // it is tied to /root/child path
}

@Location("/child")
class Child  // this is tied to just /child

@Location("/child2")
class Child2(val outer: Outer) // -> /root/child2

The cause of the second change is that objects should be symmetric with classes, and one can't add a constructor property to an object. The unfortunate consequence is that one can't write like this anymore:

@Location("/api")
object Api {
    @Location("/info")
    object Info 
}

Should be migrated to the following:

@Location("/api")
object Api {
    @Location("/info")
    class Info(val api: Api = Api)
    // the api parameter is required, weird isn't it
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/571

Hello !
Does Ktor support Kotlin Native for now (using Netty)? I saw few comments in JetBrains Blog, but can't find any detailed information about it.
Thank you !

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/319

The rational behind this proposal is that a production deployment of an application shall typically run with INFO threshold on its logs (DEBUG and TRACE are typically used only for debugging of certain subsystems) and we expect that all requests shall be logged in a production deployment by default, since the log of requests represents the starting point for investigation of any production incidents.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1989

Subsystem
Server

Is your feature request related to a problem? Please describe.
I stumbled upon a problem where I need to allow origin by wildcard or pattern, ex. http://*.example.com so any subdomain is allowed.

Describe the solution you'd like
I want to be able to specify a host with a pattern matching like regex or glob style or give us a way to set a custom callback for checking the origin

Motivation to include to ktor
There is no way to solve this with the current CORS feature.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1652

Hi all
When I run my Ktor application with gradle run then I've got the following exception:

19:21:11.795 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 11
19:21:11.811 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled
        at io.netty.util.internal.ReflectionUtil.trySetAccessible(ReflectionUtil.java:31)
        at io.netty.util.internal.PlatformDependent0$4.run(PlatformDependent0.java:225)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:219)
        at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
        at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
        at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
19:21:11.815 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
java.lang.IllegalAccessException: class io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @557caf28
        at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
        at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
        at java.base/java.lang.reflect.Method.invoke(Method.java:558)
        at io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:326)
        at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
        at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
        at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
        at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)

the content of build.gradle.kts file

plugins {
    application
    kotlin("jvm") version "1.3.61"
}

group = "io.flatmap"
version = "1.0-SNAPSHOT"


val ktor_version = "1.3.0"

repositories {
    mavenCentral()
    jcenter()
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))
    compile("io.ktor:ktor-server-netty:$ktor_version")
    compile("io.ktor:ktor-server-core:$ktor_version")
    compile("ch.qos.logback:logback-classic:1.2.3")
    testCompile(group = "junit", name = "junit", version = "4.12")
}

tasks {
    compileKotlin {
        kotlinOptions.jvmTarget = "11"
    }
    compileTestKotlin {
        kotlinOptions.jvmTarget = "11"
    }
}

application {
    mainClassName = "io.ktor.server.netty.EngineMain"
}

I am using Zulu OpenJDK 11:

 java --version
openjdk 11.0.6 2020-01-14 LTS
OpenJDK Runtime Environment Zulu11.37+17-CA (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.37+17-CA (build 11.0.6+10-LTS, mixed mode) 

I thought, this is solved through https://github.com/ktorio/ktor/issues/1190.

Thanks

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/680

Currently, it's installed in the ApplicationCallPipeline.Features phase. This is fine for route handlers that want to access call.request.origin.remoteHost, etc, but is problematic for other features that might want to use the info that XFHS provides.

As an example, I'm writing a feature that captures request metadata and puts it in a ThreadLocal (via asContextElement) so that when writing to GCP's logging service, the HttpRequest element can be populated automatically for any logging calls that are done in the coroutine context of a request. Naturally, it would be nice to populate the actual client ip here.

The Monitoring phase sounds like what I want from the docs:

Phase for tracing calls, useful for logging, metrics, error handling and so on

However, if I intercept there, I won't get to use the data from XFHS. My current workaround is to intercept in Features as the last feature I add there so that the changes applied by XFHS are visible, but that's not ideal: now logging done by features in Monitoring won't have their log entries enriched with request metadata at all.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1933

Versions: JVM OpenJDK 11
OS: Dockerized Ubuntu ARM64, Raspberry Pi 4
Ktor: 1.3.2
Kotlin: 1.3.72

Code here, docker-compose here, nginx conf here.

when using call.respond() it works fine. Why call.respondOutputStream() fails?

EDIT: After digging in Nginx debug logs it appears that:

So the upstream server returns a response with "Transfer-Encoding: chunked", and the first byte of the response body is 0x89. This character is outside of the ASCII range and clearly is not a hexadecimal number as required by the ​chunked encoding.
Clearly this is an incorrect response from the backend. Depending on how the framework you are using is expected to work, this may either indicate a bug in the framework, or you are using the framework incorrectly.

Cross reference ticket from Nginx issue board.

Nginx error:

upstream sent invalid chunked response while reading upstream

Error stack:

ERROR ktor.application - 200 OK: (request error: java.lang.IllegalStateException: servletRequest.method must not be null)
java.lang.IndexOutOfBoundsException: null
	at javax.imageio.stream.FileCacheImageOutputStream.seek(FileCacheImageOutputStream.java:170)
	at javax.imageio.stream.FileCacheImageOutputStream.close(FileCacheImageOutputStream.java:231)
	at javax.imageio.ImageIO.write(ImageIO.java:1590)
...
Suppressed: java.io.IOException: io.ktor.util.cio.ChannelWriteException: Failed to write to servlet async stream
		at io.ktor.utils.io.jvm.javaio.OutputAdapter.close(Blocking.kt:122)
		at kotlin.io.CloseableKt.closeFinally(Closeable.kt:56)
		... 8 common frames omitted
	Caused by: io.ktor.util.cio.ChannelWriteException: Failed to write to servlet async stream
		at io.ktor.server.servlet.ServletWriter.wrapException(ServletWriter.kt:119)
		at io.ktor.server.servlet.ServletWriter.onError(ServletWriter.kt:112)
		at io.ktor.server.servlet.ServletWriter.run(ServletWriter.kt:52)
		at io.ktor.server.servlet.ServletWriter$run$1.invokeSuspend(ServletWriter.kt)
		at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
...
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
		at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:309)
		at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:272)
		at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:118)
		at io.ktor.server.servlet.ServletWriter.loop(ServletWriter.kt:87)
		at io.ktor.server.servlet.ServletWriter$loop$1.invokeSuspend(ServletWriter.kt)
		... 46 common frames omitted
	Caused by: java.io.IOException: Broken pipe
		at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
		at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
		at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
		at sun.nio.ch.IOUtil.write(IOUtil.java:65)
		at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:468)
		at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:138)
...

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/294

I've set up ContentNegotiation with Gson but had troubles figuring out how I can receive POST data.

Examples pointed me towards call.receive<ValuesMap>() which doesn't work with JSON in the request body, instead it worked after creating a custom data class and mapping it directly into it. It would be nice to be able to directly map into a ValuesMap. I guess up for discussion would be how to handle nested objects, if they are stored "as is" or with nested maps.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/711

I found an error in ktor feature using jackson. It does not handle empty body content very well.

I have this example application:

data class Han(val x: Int)

fun main(args: Array<String>) {
    val server = embeddedServer(Netty, port = 8080) {
        install(ContentNegotiation) {
            jackson {
                propertyNamingStrategy = PropertyNamingStrategy.LOWER_CAMEL_CASE
            }
        }

        routing {
            post("/demo") {
                val t = call.receive(Han::class)
            }
        }
    }
    server.start(wait = true)
}

When i send with postman an empty body with contentType: application/json to /demo it crashes with the following exception:

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (InputStreamReader); line: 1, column: 0]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4133)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3988)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3037)
	at io.ktor.jackson.JacksonConverter.convertForReceive(JacksonConverter.kt:40)
	at io.ktor.features.ContentNegotiation$Feature$install$3.doResume(ContentNegotiation.kt:95)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
	at io.ktor.features.ContentNegotiation$Feature$instal...

I think this should be handled more gracefully than this hard exception.

Thank you guys for all your effort! Ktor is awesome.

Versions:

implementation("io.ktor:ktor-server-netty:0.9.5")
    implementation("io.ktor:ktor-server-core:0.9.5")
    implementation("io.ktor:ktor-jackson:0.9.5")

with kotlin 1.2.71

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1146

Ktor Version

1.2.0

Ktor Engine Used(client or server and name)

CIO

JVM Version, Operating System and Relevant Context

JDK 9, macOS, io.ktor:ktor-jackson:1.2.0

Feedback

The docs (in code) suggest that when calling receive<T> to transform a request POST body to a data class type, if the content cannot be transformed, a ContentTransformationException will be thrown.

When using io.ktor:ktor-jackson:1.2.0, instead, a Jackson Kotlin exception is thrown of type JsonMappingException (or, specifically in my case, a MissingKotlinParameterException).

Here's some sample code that I've simplified slightly to echo the request body back, and it includes the use of a thin wrapper in my application (RequestHandling). The expected behaviour is that the user would get a HttpStatusCode.BadRequest back if they submitted a request that didn't include the password field (for example, { "email": "someone@somewhere.com" }). Instead, a MissingKotlinParameterException is unhandled, forcing a 500 response. The same happens for receieveOrNull.

class SignUpHandler: RequestHandling {

    data class SignUpRequest(val email: String, val password: String)

    override suspend fun handle(call: ApplicationCall) {
        val requestBody = try {
            call.receive<SignUpRequest>()
        } catch (exception: ContentTransformationException) {
            return call.respond(HttpStatusCode.BadRequest)
        }

        return call.respond(requestBody)
    }
}

Please let me know if you need any other details 👍

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1526

Subsystem
Ktor Locations Feature
Is your feature request related to a problem? Please describe.
Ktor's Location feature is a pretty big game changer and makes it easy to create and receive requests. However, for POST requests, the source of the parameter is still sourced from the query string, instead of body or form fields. e.g.

@Location("/users/login")
data class UserInfo(val username: String, val password: String)
.
.
.
post<UserInfo> {
  // Needs `username` and `password` to be set in the query params!
}

Describe the solution you'd like
It would be nice if we could specify the source of the parameters in the declaration of the parameters itself, using annotations:

@Location("/users/login")
data class UserInfo(@Form val username: String, @Form val password: String)
.
.
.
post<UserInfo> {
  // yay it.username and it.password can be set via multipart form
}

or

@Location("/users/login")
data class UserInfo(@Body val username: String, @Body val password: String)
.
.
.
post<UserInfo> {
  // yay it.username and it.password can be set via body
}

Creating the request to be used by the ktor clients for post requests can also be specified this way.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/222

This is possible but couples the Features top much. There should be a way top allow accessing vals from one feature in another one without top tight coupling.

A possible Use case are eg. the metrics feature and the sessions feature to allow metrics on the number of sessions.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1736

Ktor Version and Engine Used (client or server and name)
1.3.2 server

Describe the bug
Using nested Feature's interceptPipeline leads to different results when using insertPhaseBefore and insertPhaseAfter.
Example:
customA { customB(1) { get { call.respond("") } } }

Both added to Child of a Route via
pipeline.insertPhaseBefore(ApplicationCallPipeline.Features, customA/B) pipeline.intercept(customA/B)

Using after:
[Phase('Setup'), Phase('Monitoring'), Phase('Features'), Phase('customB'), Phase('Call'), Phase('Fallback')]
[Phase('Setup'), Phase('Monitoring'), Phase('Features'), Phase('customA'), Phase('Call'), Phase('Fallback')]

Using before
[Phase('Setup'), Phase('Monitoring'), Phase('customA'), Phase('Features'), Phase('Call'), Phase('Fallback')]
[Phase('Setup'), Phase('Monitoring'), Phase('customB'), Phase('Features'), Phase('Call'), Phase('Fallback')]

Expected behavior
I would expect that both variants put Phase(customA) first, then Phase(customB) when merging the pipelines.

Upon googling I found the similar issue here.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1451

I've noticed that (even in hello world projects)

• After starting the server, it takes 4-5 seconds for it to start accepting API calls on localhost
• Upon stopping the server, the process often lingers around for another 5-6 seconds showing the below icon 👇

Now this may not sound like much, but starting and stopping every day hundreds of times, this could easily get very annoying. (This is on a 2019 i9 4.5Ghz Macbook Pro)

So I wonder, if I'm missing something or whether this is something that everyone faces? Is there a way to fix this?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/925

Ktor Version

1.1.2

Ktor Engine Used(client or server and name)

netty

JVM Version, Operating System and Relevant Context

java version 10.0.2
mac os

Feedback

i built a small app to test mustache with ktor but when i try to post data using html form in mustache file .hbs file
the post gave me an

415 Unsupported Media Type: POST
when i try to recieve a typed json
val param = call.receive<Quote>()

and also give me a null value when i try
val param = call.receive<Parameters>() val author = param["author"]!! val qouteText = param["quoteText"]!!

and this is my html form

` <form role="form" class="col-xs-3 text-center" style="width: 30%" action="/quotes" method="post" >

        <div class="form-group">
            <label for="quote">Quote</label>
            <br>
            <textarea class="form-control" id="quote" rows="3" required="required"></textarea>
        </div>
        <div class="text-center">
            <button type="submit" class="btn btn-primary mb-2 text-center">Confirm</button>
        </div>


    </form>`

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1763

Using ktor 1.3.1 with jetty, metrics and jackson, Ktor would appear to show a 2xx response in access logs but actually returned a 500 error with an empty body across a variety of endpoints. The error is intermittent and sometimes it would happen a lot and sometimes only a little.

Switching to the Netty engine makes the problem completely go away.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1279

A few times a second my screen is filled with:

2019-08-12 06:17:03.171 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:06.197 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:09.232 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:12.266 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:13.205 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:15.301 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:18.336 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:21.369 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:23.238 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /search
2019-08-12 06:17:24.404 [nioEventLoopGroup-4-1] INFO  Application - Unhandled: get - /
2019-08-12 06:17:24.761 [nioEventLoopGroup-4-1] INFO  Application - 302 Found: GET - / -> /static/index.html
2019-08-12 06:17:24.810 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/index.html
2019-08-12 06:17:24.878 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/css/reset.css
2019-08-12 06:17:24.900 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/css/mainstyle.css
2019-08-12 06:17:24.927 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/nav/nav_over_01.png
2019-08-12 06:17:24.935 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/nav/nav_05.png
2019-08-12 06:17:25.325 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/diracian.png
2019-08-12 06:17:25.340 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/alto-whitelabel.png
2019-08-12 06:17:25.356 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/andaz-whitelabel.jpg
2019-08-12 06:17:25.369 [nioEventLoopGroup-4-1] INFO  Application - 200 OK: GET - /static/images/payscout-whitelabel.png

The only difference from a request that works is the GET is lowercase and the other is uppercase. I don't see any mention in RFC 7231 that says that the verb needs to be capitalized. However, they do make a point to capitalize it through the entire document. Is there a way for me to handle these unhandled requests?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1634

Ktor Version and Engine Used (client or server and name)
1.3.0, server, ktor-auth-ldap

Describe the bug
When using the default ldap authentication any combination of a password and username gets successfully authenticated.
I use the following code as mentioned in the documentation:

basic("authName") {
        realm = "realm"
        validate { credential ->
            ldapAuthenticate(credential, "...", "...")
        }
    }

I used the following to work around this problem:

basic("ldap") {
            validate { credential ->
                val ldapUrl = "..."
                val configure: (MutableMap<String, Any?>) -> Unit = { env ->
                    env["java.naming.security.principal"] = credential.name
                    env["java.naming.security.credentials"] = credential.password
                    env["java.naming.security.authentication"] = "simple"
                }
                ldapAuthenticate(credential, ldapUrl, configure) {
                    UserIdPrincipal(credential.name)
                }
            }
        }

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1698

Initial report — https://youtrack.jetbrains.com/issue/KT-36659

Ktor Version and Engine Used (client or server and name)
Ktor 1.3.1 JS/JVM (Netty)

Describe the bug
In a multiplatform project, Ktor can't find a resource file (module-name/build/distributions/module-name.js) in a child module.

To Reproduce
Steps to reproduce the behavior:

1. Open the attached file
2. Run ./gradlew run
3. Ktor can't find the resource file

Expected behavior
Ktor should find the file and process it.

Screenshots

KT-36659.zip

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1469

ktor 1.2.6
server is io.ktor:ktor-server-cio

I have exception if try check open server port over nmap

Steps:

1. Start server
2. Scan server use nmap... "nmap -P0 DEVICE_IP -p 8999"

2019-12-02 17:55:34.917 9591-9669/? E/AndroidRuntime: FATAL EXCEPTION: DefaultDispatcher-worker-3
Process: org.jajaz.wearreader, PID: 9591
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:419)
at kotlinx.io.nio.ChannelsKt.read(Channels.kt:117)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:75)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invoke(Unknown Source:10)
at kotlinx.coroutines.io.ByteBufferChannel.writeSuspendSession(ByteBufferChannel.kt:1931)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1.invokeSuspend(CIOReader.kt:65)
at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1.invoke(Unknown Source:10)
at kotlinx.coroutines.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:123)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedKt.resumeCancellable(Dispatched.kt:457)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch(Unknown Source:1)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:47)
at kotlinx.coroutines.BuildersKt.launch$default(Unknown Source:1)
at kotlinx.coroutines.io.CoroutinesKt.launchChannel(Coroutines.kt:118)
at kotlinx.coroutines.io.CoroutinesKt.writer(Coroutines.kt:76)
at io.ktor.network.sockets.CIOReaderKt.attachForReadingDirectImpl(CIOReader.kt:61)
at io.ktor.network.sockets.NIOSocketImpl$attachForReading$1.invoke(NIOSocket.kt:45)
at io.ktor.network.sockets.NIOSocketImpl$attachForReading$1.invoke(NIOSocket.kt:18)
at io.ktor.network.sockets.NIOSocketImpl.attachFor(NIOSocket.kt:80)
at io.ktor.network.sockets.NIOSocketImpl.attachForReading(NIOSocket.kt:41)
at io.ktor.network.sockets.SocketsKt.openReadChannel(Sockets.kt:111)
at io.ktor.server.cio.HttpServerKt$httpServer$acceptJob$1.invokeSuspend(HttpServer.kt:109)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1391

Ktor Version and Engine Used (client or server and name)
versions: 1.2.4, 1.2.5
modules: server-netty, jackson, auth, locations

Describe the bug
In seemingly random intervals I'm seeing batches of logs /route-name, channel failed. I've tracked the source of the log to [here].(https://github.com/ktorio/ktor/blob/424d1d2cfaa3281302c60af9500f738c8c2fc846/ktor-server/ktor-server-host-common/jvm/src/io/ktor/server/engine/DefaultEnginePipeline.kt#L103)
I'm unable to reproduce the behavior on my local machine. That makes me belive it might happen due to high load. Since the message does not offer much insight to what is happening, I'm unable to properly analyse the problem.

To Reproduce
Not sure

Expected behavior
Clear message what is happening.

Screenshots

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1298

Ktor Version and Engine Used (client or server and name)
Ktor 1.2.3 + Netty

Describe the bug
Hello, maybe this is not a bug, and I just don’t understand how this should work, but in my opinion there may be a mistake.

I create a server with this code (imagine that debugPlay does some work for +-200 ms):

fun main() {
val env = applicationEngineEnvironment {
        // Public API
        connector {
            port = 1717
        }
        module {
            mainGet()
        }
    }
    embeddedServer(Netty, environment = env, configure = {
        connectionGroupSize = 1
        workerGroupSize = 1
        callGroupSize = 1
        runningLimit = 1
        requestQueueLimit = 1
    }).start(true)
}
}

fun Application.mainGet() {
    install(CallLogging)
    routing {
        get("/api/checkThis") {
            call.respond(TextContent("""{"msg": "unaccessable when server process old queue request"}""", ContentType.Application.Json))
        }
        get("/api/debugPlay") {
            Thread.sleep(200)
            println("debugPlay")
            call.respond(TextContent(JSONObject().put("sign", "123").toString(), ContentType.Application.Json))
            return@get
       }
    }
}

I set all available configure options to 1 to demonstrate the problem.
So, I expect that the queue can save no more than one request. Next, I send 1000 requests from another application simultaneously and close it without waiting for an answer.

However, Ktor continues to respond to requests for a long time, despite the fact that only one request should remain in the queue. Here is the log:

debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
debugPlay
[nioEventLoopGroup-4-1] INFO Application - 200 OK: GET - /api/debugPlay, cancelled
...

If I do not install Netty options and just use the standard configuration, then the server stops responding to other requests (to another route), and continues to process old ones that have already been canceled. Thus, it is very easy to make a ddos ​​attack by sending a lot of requests to which the server will respond, while ignoring new requests from real users. Can you help: is saving old requests in the queue an error (since I set a limit of 1 request), or I just don’t understand why it is so right? And how to optimally configure ktor so as not to become an easy target for a ddos attack?)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1296

Ktor Version and Engine Used (client or server and name)

implementation("io.ktor:ktor-locations:1.2.3")
implementation("io.ktor:ktor-serialization:1.2.3")

Describe the bug
Default data conversions fails when nesting Object -> Class -> Object -> Class with the @Location annotation like in the following example (Branch is an object, List a class):

val branchList = Organization.ById.Branch.List(Organization.ById("abc"))
// Would become "/organizations/abc/branches"
val branchListHref = application.locations.href(organizationBranchList)

It yields the following error:

Type class Organization$ById is not supported in default data conversion service
io.ktor.util.DataConversionException: Type class Organization$ById is not supported in default data conversion service
	at io.ktor.util.DefaultConversionService.toValues(ConversionService.kt:46)
	at io.ktor.locations.Locations.pathAndQuery(Locations.kt:210)
	at io.ktor.locations.Locations.href(Locations.kt:222)
	at ...ApplicationTest$branch offices for organization$1.invoke(ApplicationTest.kt:183)

A similar version using nested Object -> Class -> Class works like expected (BranchList is a class):

val branchList = Organization.ById.BranchList(Organization.ById("abc"))
val branchListHref = application.locations.href(organizationBranchList)

Both are based on the following declaration of API routes:

@Location("/organizations")
object Organization {
    @Location("/")
    class List(val offset: Int = 0, val limit: Int = 20)
    @Location("/{id}")
    class ById(val id: String) {
        // 1)  Does not work
        @Location("/branches")
        object Branch {
            @Location("/")
            class List(val organization: Organization.ById, val offset: Int = 0, val limit: Int = 20)
            @Location("/{id}")
            class ById(val organization: Organization.ById, val id: String)
        }

        // 2) Works 
        @Location("/branches/")
        class BranchList(val organization: ById, val offset: Int = 0, val limit: Int = 20)
        @Location("/branches/{id}")
        class BranchById(val organization: ById, val id: String)
    }
}

Style 2) seems more idiomatic as it allows to POST or PUT to Organization.ById.Branches (/organizations/123/branches) to create a new Branch.

To Reproduce
See above code example.

Expected behavior
Arbitrary nesting of locations should work.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1191

Ktor Version

1.2.1

Ktor Engine Used (client or server and name)

Netty

JVM Version, Operating System and Relevant Context

JDK 8, Mac

Issue

I'm building a multi-layer custom authentication for my web service. I declare it like this:

// in Application.module()
install(Authentication) {
  appId(AUTH_APP_ID) { // this one is my custom one
    validate {
      if (AuthorizedApps.isReadAuthorized(it.appId)) // simplified logic
        AppIdPrincipal(AuthorizedApps.getAppName(it.appId)!!)
      else null
    }
  }
}

And routes look like this:

routing {
  authenticate(AUTH_APP_ID) {
    get("/info", controller<ServiceInformationController>()) // controller handles everything
  }
}

So... this works as expected. It was actually much easier than anticipated.

THE PROBLEM

Now I want to add another custom authentication, for example "AppSecret". Same as ID, just handles a different header (app_secret) along with this app_id header.

I won't paste the whole thing here (see the bottom for more info), but it's pretty much the same.. except for this part:

authenticate(AUTH_APP_ID, AUTH_APP_SECRET) { // observe: two authentication methods
  get("/info", controller<ServiceInformationController>())
}

I also tried the other approach, with nested authenticate blocks.

In both cases, the second authentication method is ignored.

The truth is.. yes, right now, I only need the one (secret), but let's say in the future I want to add another one, that I want to combine with the others.. for example JWT or something on top of AppID/AppSecret combo.

Why is my second authentication method completely ignored?
I debugged this, Ktor never even tries after the first auth method is successful.

Here's my custom authentication for reference (AppID):

package tokenizer.v1.security

import io.ktor.application.ApplicationCall
import io.ktor.application.call
import io.ktor.auth.Authentication
import io.ktor.auth.AuthenticationFunction
import io.ktor.auth.AuthenticationPipeline
import io.ktor.auth.AuthenticationProvider
import io.ktor.auth.Credential
import io.ktor.auth.Principal
import io.ktor.http.HttpStatusCode
import io.ktor.request.ApplicationRequest
import io.ktor.request.header
import io.ktor.response.respond
import tokenizer.v1.Configuration.Headers
import tokenizer.v1.security.AppIdAuthenticationProvider.Configuration

data class AppIdCredential(val appId: String) : Credential

data class AppIdPrincipal(val appName: String) : Principal

class AppIdAuthenticationProvider(configuration: Configuration) : AuthenticationProvider(configuration) {
  class Configuration(name: String?) : AuthenticationProvider.Configuration(name) {
    // provides a validation function that will check given `AppIdCredential` instance
    // and return `Principal` (or `null` if credential does not correspond to an authenticated principal)
    var authenticationFunction: AuthenticationFunction<AppIdCredential> = { null }

    fun validate(body: suspend ApplicationCall.(AppIdCredential) -> Principal?) {
      authenticationFunction = body
    }
  }

  val authenticationFunction = configuration.authenticationFunction
}

// installs an app ID authentication mechanism
fun Authentication.Configuration.appId(name: String? = null, configure: Configuration.() -> Unit) {
  val provider = AppIdAuthenticationProvider(Configuration(name).apply(configure))
  val authenticateFunction = provider.authenticationFunction

  provider.pipeline.intercept(AuthenticationPipeline.RequestAuthentication) { context ->
    val credential = call.request.appIdAuthenticationCredential()
    val principal = credential?.let { authenticateFunction(call, it) }

    val cause = when {
      credential == null -> call.respond(HttpStatusCode.Forbidden)
      principal == null -> call.respond(HttpStatusCode.Unauthorized)
      else -> null
    }

    if (cause != null) {
      // maybe respond with something? probably not...
    }

    principal?.let { context.principal(it) }
  }

  register(provider)
}

// retrieves App ID Credential for this `ApplicationRequest`
@Suppress("MoveVariableDeclarationIntoWhen")
fun ApplicationRequest.appIdAuthenticationCredential(): AppIdCredential? {
  val appId = header(Headers.APP_ID)
  // we need a valid app ID
  if (appId.isNullOrBlank()) return null
  // we have "something", so let's use this
  return AppIdCredential(appId)
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1052

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

Java 1.8.0_181 Is what my JAVA_HOME is set to.

Also:

IntelliJ IDEA 2019.1 (Ultimate Edition)
Build #IU-191.6183.87, built on March 27, 2019
JRE: 1.8.0_202-release-1483-b39 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.3

Feedback

I'm seeing the following error in the browser console (chrome), while serving the page locally on my computer: Failed to load resource: net::ERR_INCOMPLETE_CHUNKED_ENCODING

That's all the information with the error. This is the first time I've seen this. It's just another indicator that there seems to be some race condition with the ktor code. Again, I can't reproduce this.

Here is the video that I saw the error happen with while loading. I have to zip it so github accepts it:
intersection-time-lapse.mp4.zip

The video is loaded via this html element:

    <video id="home-background-video" muted playsinline autoplay loop preload="auto" poster="assets/images/intersection-first-frame.jpg" width="1920" height="1080">
      <source src="assets/images/intersection-time-lapse.mp4" type="video/mp4">
    </video>

The video is served via this ktor server code, via the static files route:

fun main() {
    embeddedServer(
        Netty,
        port = 8080,
        module = Application::main
    ).apply { start(wait = true) }

}

private fun Application.main() {
    features()
    registration()
    staticFiles()
}

private fun Application.features() {
    install(Compression)
    install(ContentNegotiation) {
        jackson {
            configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
            configure(SerializationFeature.INDENT_OUTPUT, false)
        }
    }
}

private fun Application.staticFiles() {
    routing {
        static {
            files("web")
            default("web/index.html")
        }
    }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/990

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Server

JVM Version, Operating System and Relevant Context

Java 1.8.0_161, macOS Mojave (10.14.2)

Feedback

When using AES 256 with SessionTransportTransformerEncrypt.kt it throws java.security.InvalidAlgorithmParameterException: Wrong IV length: must be 16 bytes long.

It appears that the IV generator uses the encoded key length which is 32 Bytes for AES 256, however, AES 256 requires a 16 byte IV.

Current work-around:

transform(
    SessionTransportTransformerEncrypt(
        sessionSecretKey,
        sessionHmacKey,
        ivGenerator = {
            SecureRandom().generateSeed(16)
        }
    )
)

Proposed Solutions:

• Change the implementation to check if the key length is AES and greater than 16, default to 16
• Add a check in init block for AES 256 and check IV length prior to passing it to encrypt method. If IV length is greater than 16, throw an error that says user must provide their own IV Generator that generates a 16 byte IV when using AES 256.
• Add a comment to the class that warns users to provide their own ivGenerator if using AES 256.

Please let me know if you have any questions or if I missed something.

Thanks for your help!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1001

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

Java 1.8.0_172, OSX / Linux

Feedback

Given the following server using Jackson content negotiation: https://gist.github.com/mattrussell-sonocent/faa63a417fd3b3f7424de93d4a5e7d1f

We're finding that slow client POSTs can block other requests from being processed. For example, if we telnet localhost 8080 and begin a request:

POST /login HTTP/1.1
Content-Length: 6500
Content-Type: application/json

{

Then attempt wget http://localhost:8080/, the server doesn't respond until the telnet POST session is completed or terminated. (On a dev Macbook, I had to send ~8 GET requests to start seeing this behaviour. In a container environment, it happened on the first request.)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/850

Ktor Version

Version: 1.1.1

Ktor Engine Used(client or server and name)

Server and am following this here:
https://ktor.io/servers/calls/responses.html

JVM Version, Operating System and Relevant Context

JVM: 1.8
OS: Windows 10
Working in IntelliJ Ultimate 2018.1 within a gradle project
The whole project builds, but this line continues to be an error while coding.

Below is a picture of what is happening:
https://i.imgur.com/GE8n7oX.jpg

A picture of my project hierarchy:
https://i.imgur.com/jKngO0i.jpg

All the code from my main class named, "App.kt":

import com.jcabi.ssh.Shell
import com.jcabi.ssh.Ssh
import java.io.File
import org.apache.commons.io.FileUtils
import io.ktor.application.*
import io.ktor.features.*
import io.ktor.http.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.application.*

class App {
    val greeting: String
        get() {
            return "Hello world."
        }
}

fun Application.module() {
    routing {
        get("/") {
            call.respondText("Hehe")
        }
    }
}

fun main(args: Array<String>) {
    embeddedServer(Netty, 8080, watchPaths = listOf("AppKt"), module = Application::module).start()
}

Feedback

I really would like to use Ktor for what I am going to make and if this can be fixed, that would be amazing. Thanks in advance.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1445

Ktor Version and Engine Used (client or server and name)
version, client/server, additional ktor modules included

Describe the bug
Jackson cant serialize a json into a model on Get but it works on Post with the exact same code.

route("/auth") {
                get() { //throw error at runtime
                    val post = call.receive<PostUser>()
                    call.respond(HttpStatusCode.Created)
                }
                post() { //works fine
                    val post = call.receive<PostUser>()
                    call.respond(HttpStatusCode.Created)
                }
            }

Expected behavior
Ability to serialize body json in get method.

By sending in postman:
GET http://0.0.0.0:8080/users/auth
in raw json

{
	"user": {
		"id": "767e5d91-1f84-4761-9fe6-1efe176d0d7a",
		"username": "typescripst",
		"password": "strddingentz"
	}
}

I get the following error (unlike with post for the same request)

2019-11-17 00:15:30.104 [nioEventLoopGroup-4-1] ERROR Application - Unhandled: GET - /users/auth
com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (InputStreamReader); line: 1, column: 0]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4145)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4000)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3049)
	at io.ktor.jackson.JacksonConverter.convertForReceive(JacksonConverter.kt:44)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:153)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:118)
	at com.back.ApplicationKt$module$5$2$3$1.invokeSuspend(Application.kt:109)
	at com.back.ApplicationKt$module$5$2$3$1.invoke(Application.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:98)
	at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:180)
	at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:97)
	at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:137)
	at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:106)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
	at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:36)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:26)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:20)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:515)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

ps:

I've installed jackson with

install(DataConversion)
install(ContentNegotiation) {
        jackson {
            enable(SerializationFeature.INDENT_OUTPUT)
        }
    }

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1444

Ktor 1.2.3 – ktor-server-session

Describe the bug
io.ktor.sessions.CacheStorage.write() throws cached exception

To Reproduce
Steps to reproduce the behavior:

1. var cachedStorage = CacheStorage(delegateStorage,...)
2. cachedStorage.read("myid",...) – this will throw NoSuchElementException from delegateStorage
3. Subsequent cachedStorage.read("myId",.. ) will throw cached NoSuchElementException as acrually caches kotlinx.coroutines.Deferred<T> – this is desired behaviour
4. But calling cachedStorage.write("myId",..) or cachedStorage.invalidate("myId",...) also thows that cached exception. This breaks ktor session behavior, confuses users and makes them reload page multiple times.

Expected behavior
Just ignore cached exception for invalidate & write

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1401

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.2.5

Describe the bug
I use not custom port for http server (port 8999). When i use "call.url" method inside "action" for form i get next link - "http://localhost/login" without port.
The function "call.respondRedirect" behaves similarly!
When i use "io.ktor:ktor-server-netty-1.2.5" i no have problem

Example:

get<Login> {
        call.respondDefaultHtml2(emptyList(), CacheControl.Visibility.Public, title = Localization.titleLogin) {
            h2 { +Localization.titleLogin }
            form(
                action = call.url(Login(initPassword)),
                encType = FormEncType.applicationXWwwFormUrlEncoded,
                method = FormMethod.post)
            {
                div {
                    +Localization.lbPassword
                    passwordInput(name = "password") { autoFocus = true}
                }
                br
                div {
                    submitInput(classes = "pure-button pure-button-primary") {
                        value = Localization.btLogin
                    }
                }
            }
        }
    }

For fix i use "locations.href()" method.

Also for fix "call.respondRedirect(Index())" i use "call.respondRedirect(locations.href(Index()))"

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1369

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-netty:1.2.5

Describe the bug
Ktor not work on Android API =< 25

To Reproduce
When i try to create server i get error:

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.myapplication, PID: 5399
    java.lang.NoClassDefFoundError: Failed resolution of: Ljava/time/ZoneId;
        at io.ktor.features.DefaultHeaders.<clinit>(DefaultHeaders.kt:70)

ZoneId was added in android just in API 26

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1161

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

OpenJDK 1.8.0 / Linux

Feedback

The documentation around config at https://ktor.io/servers/configuration.html#accessing-config does not seem to work if we use the "-config" command line option.

We generally like to use environment specific configuration files, managed separate from the code base. Using the -config command line option to provide a file seems appropriate, but we are encountering issues with any code relying on this style of config access:

val config = HoconApplicationConfig(ConfigFactory.load())

This seems not to include the configuration provided on the command line. The documentation also mentions configuration access via something like this:

val port: String = application.environment.config
    .propertyOrNull("ktor.deployment.port")?.getString()
    ?: "80"

This approach fails for us in that we haven't worked out where that initial application object would come from.

Preferably we would like to see something where a typed configuration is broken into parts and passed around, such as done in Dropwizard or Config4k. For example: if we have more than one database connection, a DatabaseConfig type that appears more than once helps keeping the code clean and fails fast, vs. something that is based on prefixes and assumptions.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1057

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

1.8 MacOS 10.13.6
Intel i5
16gb RAM

Feedback

If i siege single endpoint on ktor, e.g. from ktor generator

        get("/json/jackson") {
            call.respond(mapOf("hello" to "world"))
        }

netty(i think its netty's problem not ktor tbh) stops responding after ~16k requests using siege

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1046

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

CIO (Server)

JVM Version, Operating System and Relevant Context

Tested with:

• OSX 10.14.4/Zulu 8.31.0.1-macosx
• Docker image: adoptopenjdk/openjdk8-openj9

Feedback

Reading from the receive channel causes all cores to reach 100% CPU usage.
I have only experienced this issue with the CIO engine.

Minimal example for reproducing:

embeddedServer(CIO, port = 8080) {
    routing {
        post("/cpu") {
            val channel = call.request.receiveChannel()
            val buffer = ByteBuffer.allocate(1024 * 64)
            while (!channel.isClosedForRead) {
                channel.readAvailable(buffer)
                buffer.clear()
            }
            call.respondText("Bye")
        }

        post("/multipart") {
            val multipart = call.receiveMultipart()
            multipart.readAllParts()
            call.respondText("Bye")
        }
    }
}.start(wait = true)

Both of these endpoints trigger massive CPU usage.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1229

Subsystem
Server, ktor-auth

Is your feature request related to a problem? Please describe.
Request support for challenge builder function to DigestAuth

Describe the solution you'd like
Same as JWTAuthChallengeFunction but for DigestAuth

Motivation to include to ktor
My project uses DigestAuth and I hope to use this feature. Thank you for any consideration.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1552

Subsystem
Ssl part of Application.conf or the SslConnector if that is easiest to start with

Is your feature request related to a problem? Please describe.
I did a security test of my backend recently and it brought up that I don't support TSL 1.3 (which is supported in Netty; https://stackoverflow.com/questions/53406502/how-to-enable-tlsv1-3-in-netty) and I support TSL 1.0 which is outdated and unsecure.
This feels bad.

Describe the solution you'd like
I want to disable TSL 1.0 & enable TSL 1.3 (I really want to disable TSL 1.0)

Motivation to include to ktor
Security is always key, the documentation is good for now but this part is not clear & I can't find a way to achieve this. It'd be great to check this boxes :ballot_box_with_check:

Thanks for an awesome framework!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1504

Compression Feature can not support stream response, response data at call.respond done every time.

install(Compression)

call.respond(object : OutgoingContent.WriteChannelContent() {
            override val contentType = ContentType("application", "stream+json")
            override val status: HttpStatusCode? = HttpStatusCode.OK

            override suspend fun writeTo(channel: ByteWriteChannel) {
                channel.writeLong(1L)
                channel.flush()
                delay(2000)
                channel.writeLong(2L)
                channel.flush()
            }
}

Expect the same as not enabling compression, invoke flush method can send chunk response.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/758

When stopping the server in this test for my custom feature to handle single page applications strange errors shows up and idk ho to handle them:

17:36:27.091 [Test worker @coroutine#1] INFO ktor.application - No ktor.deployment.watch patterns specified, automatic reload is not active
17:36:27.123 [Test worker @coroutine#1] INFO ktor.application - Responding at http://127.0.0.1:8080
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol init
INFORMAZIONI: Initializing ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector
INFORMAZIONI: Using a shared selector for servlet write/read
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardService startInternal
INFORMAZIONI: Starting service [Tomcat]
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardEngine startInternal
INFORMAZIONI: Starting Servlet Engine: Apache Tomcat/9.0.10
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol start
INFORMAZIONI: Starting ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol pause
INFORMAZIONI: Pausing ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.catalina.core.StandardService stopInternal
INFORMAZIONI: Stopping service [Tomcat]
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
AVVERTENZA: The web application [ROOT] appears to have started a thread named [DefaultDispatcher-worker-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.doPark(CoroutineScheduler.kt:845)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.cpuWorkerIdle(CoroutineScheduler.kt:823)
 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:729)
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol stop
INFORMAZIONI: Stopping ProtocolHandler ["http-nio-8080"]
nov 25, 2018 5:36:27 PM org.apache.coyote.AbstractProtocol destroy
INFORMAZIONI: Destroying ProtocolHandler ["http-nio-8080"]
BUILD SUCCESSFUL in 3s
7 actionable tasks: 5 executed, 2 up-to-date
17:36:28: Tasks execution finished ':cleanTest :test --tests "it.lamba.ktor.features.Tests.testResources"'.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/917

Ktor Version

1.1.2

Ktor Engine Used(client or server and name)

Netty

JVM Version, Operating System and Relevant Context

OpenJDK 1.8.0_191, Ubuntu 18.04.1

Feedback

I'm using jackon as an content negotiator. I'm trying to return on my endpoint as a response Flowable<> but then I get an exception saying:

"error":"com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class io.reactivex.internal.operators.flowable.FlowableFlatMap and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)"

Is there any way to return stream of data from reactive extensions as Flowable to the client properly parsed in ktor? When using Spring I had no problem doing that just by defining return type. I searched from some examples given in the docs, but they use either awaitLast or consumeEach for getting access to the internals of Flowable.

Here is sample code for better understanding of my intentions:

    routing {
        post(PROVIDERS_URL) {
            errorAware {
                logger.info("Get all job offers")
                val something= call.receive<Something>()
                call.respond(providerAdapter.acquireSomething(something))
            }
        }
    }

fun acquireSomething(something: Something): Flowable<Some>

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/677

We are using self-signed certificates. I set them up the way it's done in the tutorial.

security {
    ssl {
      keyStore = ${?KEYSTORE_LOCATION}
      keyStorePassword = ${?KEYSTORE_PASSWORD}
      privateKeyPassword = ${?KEY_PASSWORD}
    }
  }

I know for sure that all those params and the keystore exist, but it fails with very hardly traceble error
Exception in thread "main" java.lang.IllegalStateException: connector.keyStore.getCe…Chain(connector.keyAlias) must not be null at io.ktor.server.netty.NettyChannelInitializer.<init>(NettyChannelInitializer.kt:41) at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:77)
However! Jetty with exact same settings just works.
They way I start server is just embeddedServer(Netty, commandLineEnvironment(args)).start() in main method

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/901

Ktor Version

1.0.1 but I think this bug also exists in 1.1.2

Only tested with GSON, may be Jackson is also affected.

Feedback

Already noticed in #157. The nullable Kotlin type is not taken into account by the GSON deserialization.
Kotlin Data class:

data class Something(val value: String)

Data in HTTP:

{}

Result:

Something(value=null)

Actual workaround:

We have to add the nullable property on all data class fields.

Expecting result:

If any non-nullable field is not fulfilled, the deserialization should return null (or throw if <T> receiveOrNull() is not used).

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1653

Is there a way to setup auto-reloading when the contents of the resources directory change?

I'm setting up static serving with SSR (via HTML templating) and just putting all my frontend stuff in resources directory. Would be super helpful in development if my frontend changes (in the resources directory) are auto-reloaded by ktor (i.e, always serve the latest files).

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/840

Ktor Version

1.1.1

Ktor Engine Used(client or server and name)

Server

JVM Version, Operating System and Relevant Context

IntelliJ IDEA 2018.3.2 (Ultimate Edition)
Build #IU-183.4886.37, built on December 17, 2018
JRE: 1.8.0_152-release-1343-b26 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.13.6

Feedback

My application.conf appears to not be found. I'm using the multiplatform plugin with a single jvm target. I'm doing this because I'm exposing server models to android and iOS clients, and using multiplatform across all modules seemed like the easiest way to handle this. Please correct me if there is a better way to do this.

Code from Main.kt:

object EdgeApplication {
    @JvmStatic fun main(args: Array<String>) {
        embeddedServer(Netty,
            port = 8080,
            watchPaths = listOf(
                "controller",
                "data",
                "edge",
                "model",
                "persist"
            ),
            module = Application::authModule
        ).start(wait = true)
    }
}

fun Application.authModule() {
    val issuer = environment.config.property("jwt.domain").getString()
    val audience = environment.config.property("jwt.audience").getString()
    val realm = environment.config.property("jwt.realm").getString()
    // ...
}

application.conf:

jwt {
    domain = "https://jwt-provider-domain/"
    audience = "jwt-audience"
    realm = "ktor sample app"
}

Stacktrace:

/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=56173:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/tools.jar:/Users/scott.pierce/workspace/WMBA/check-in/checkin-server/edge/build/classes/kotlin/jvm/main:/Users/scott.pierce/workspace/WMBA/check-in/checkin-server/controller/build/classes/kotlin/jvm/main:/Users/scott.pierce/workspace/WMBA/check-in/checkin-server/model/build/classes/kotlin/jvm/main:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.3.11/dd9bff00d6cfca58b6c1fe89be8e0678e35cf35f/kotlin-stdlib-jdk8-1.3.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-netty/1.1.1/4ecadba38fd5f053268563f374c3970b9c5b90f7/ktor-server-netty-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-auth-jwt/1.1.1/cd4ebaa19bdaf929ed11c82f05484e7d7a8c3b52/ktor-auth-jwt-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-locations/1.1.1/a0c31f0493438032e4b3966231b2e0766047a6a8/ktor-locations-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-jackson/1.1.1/b65c7a4d75400949fc64edd5d377e2f6c5017a3d/ktor-jackson-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.9.2/e1653d338703d8233cc1ac18c6722510bdaceb4f/jackson-datatype-jsr310-2.9.2.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.3.11/4cbc5922a54376018307a731162ccaf3ef851a39/kotlin-stdlib-1.3.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.3.11/4839661cf6ce3c14b65ed7dcf5b9249b44ecca16/kotlin-stdlib-jdk7-1.3.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.3.11/aae7b33412715e9ed441934c4ffaad1bb80e9d36/kotlin-reflect-1.3.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/atomicfu/0.12.0/121e5983b99875b8c56ce57b31c915a251ab4ec0/atomicfu-0.12.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.typesafe/config/1.3.1/2cf7a6cc79732e3bdf1647d7404279900ca63eb0/config-1.3.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-jdk8/1.1.0/c4985f08c92832fd4e8b975395e4431a258f0174/kotlinx-coroutines-jdk8-1.1.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-io-jvm/0.1.3/17b417c64bbce4361514c3a70133d7cc12531f98/kotlinx-io-jvm-0.1.3.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-io-jvm/0.1.3/6103667250c1d9a27e96ad49d8ccce988f94006d/kotlinx-coroutines-io-jvm-0.1.3.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-host-common/1.1.1/25f4c2c8c5827a544f710bf8ba0dc6892902b50b/ktor-server-host-common-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.24.Final/c0c0d9d20402e4493083447052b59d5680e88b2e/netty-codec-http2-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.eclipse.jetty.alpn/alpn-api/1.1.3.v20160715/a1bf3a937f91b4c953acd13e8c9552347adc2198/alpn-api-1.1.3.v20160715.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-auth/1.1.1/8b814227032b5c1cf0e2b8874bf1e8ab8b62f29d/ktor-auth-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.auth0/java-jwt/3.4.0/e7ce62e0e190ed86c6edca4c0562dd03b432a141/java-jwt-3.4.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.auth0/jwks-rsa/0.5.0/db7f14bb8e88b28edbd7c0e7de9ebc67004d5114/jwks-rsa-0.5.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-kotlin/2.9.2/a9cf72ae68a641ade45c400b6cec2d363818f11b/jackson-module-kotlin-2.9.2.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.3.11/d8b8e746e279f1c4f5e08bc14a96b82e6bb1de02/kotlin-stdlib-common-1.3.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.1.0/20274ba816d8938fdb4d7a3806719691db60fa1e/kotlinx-coroutines-core-1.1.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/atomicfu-common/0.12.0/85d8f3e41940024524c45f9143e5e9866f7a6543/atomicfu-common-0.12.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-io/0.1.3/f3df9e1d097a0140b93c6874a756e22b7520144f/kotlinx-io-0.1.3.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-io/0.1.3/448395d0e62f1047025ccdc1a5238337460af399/kotlinx-coroutines-io-0.1.3.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-core/1.1.1/24c2f13ac14e8bd503ed7a30176f54653dc2e61/ktor-server-core-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-cio/1.1.1/60a047876b54fc6f830b20e2d76c9e2a788a02bd/ktor-http-cio-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.24.Final/8f20009953b2c7c3d860cef928007bc01aa58ac/netty-codec-http-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.24.Final/bad56e7da211c5ebe031ae155cb648b1065c7bb6/netty-handler-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-client-core-jvm/1.1.1/9306af36f11ea6ed14705f4b15cdda308599ab60/ktor-client-core-jvm-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.googlecode.json-simple/json-simple/1.1.1/c9ad4a0850ab676c5c64461a05ca524cdfff59f1/json-simple-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.9.6/cfa4f316351a91bfd95cb0644c6a2c95f52db1fc/jackson-databind-2.9.6.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.11/3acb4705652e16236558f0f4f2192cc33c3bd189/commons-codec-1.11.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/2.6/815893df5f31da2ece4040fe0a12fd44b577afaf/commons-io-2.6.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/19.0/6ce200f6b23222af3d8abb6b6459e6c44f4bb0e9/guava-19.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-common/1.1.0/d81cc72afd6e49f5270db24bae935f9bfcfd859e/kotlinx-coroutines-core-common-1.1.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-utils-jvm/1.1.1/e1efbfe2409e41b4695cdbb4fa938db602b1d5d0/ktor-utils-jvm-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-jvm/1.1.1/fe0a7e2daf1761eda32d1c8e19d5c711932b3af9/ktor-http-jvm-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-network/1.1.1/bacc1b44c6d45202a926da9488881a6373d2be73/ktor-network-1.1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.24.Final/290857e5103956bbda11836e33245f2439226b77/netty-codec-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/junit/junit/4.10/e4f1766ce7404a08f45d859fb9c226fc9e41a861/junit-4.10.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.9.0/7c10d545325e3a6e72e06381afe469fd40eb701/jackson-annotations-2.9.0.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.9.6/4e393793c37c77e042ccc7be5a914ae39251b365/jackson-core-2.9.6.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.24.Final/d37292c94d3a4cba48d9b6cfb6e8e55282035d0d/netty-transport-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.1/860340562250678d1a344907ac75754e259cdb14/hamcrest-core-1.1.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.24.Final/e354bed2e60b568307138e403f55ba241c1c16d2/netty-buffer-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.24.Final/dbc1e5b50d31aed883ea3beeb6489e1977d0687f/netty-resolver-4.1.24.Final.jar:/Users/scott.pierce/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.24.Final/7eeecd7906543214c3c1c984d275d3c6de10b99d/netty-common-4.1.24.Final.jar com.wmba.checkin.edge.EdgeApplication
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at kotlin.reflect.jvm.internal.calls.CallerImpl$Method.callMethod(CallerImpl.kt:71)
	at kotlin.reflect.jvm.internal.calls.CallerImpl$Method$Static.call(CallerImpl.kt:80)
	at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:106)
	at kotlin.reflect.jvm.internal.KCallableImpl.callDefaultMethod$kotlin_reflect_api(KCallableImpl.kt:152)
	at kotlin.reflect.jvm.internal.KCallableImpl.callBy(KCallableImpl.kt:110)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.callFunctionWithInjection(ApplicationEngineEnvironmentReloading.kt:347)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:297)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:273)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:126)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:245)
	at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:106)
	at com.wmba.checkin.edge.EdgeApplication.main(Main.kt:45)
Caused by: io.ktor.config.ApplicationConfigurationException: Property jwt.domain not found.
	at io.ktor.config.MapApplicationConfig.property(MapApplicationConfig.kt:48)
	at com.wmba.checkin.edge.MainKt.authModule(Main.kt:50)
	... 16 more

build.gradle:

apply plugin: 'org.jetbrains.kotlin.multiplatform'

kotlin {
    targets {
        fromPreset(presets.jvm, 'jvm')
    }

    sourceSets {
        jvmMain {
            dependencies {
                implementation deps.kotlin.stdlib.jvm
                
                implementation project(':controller')
                api project(':model')
                
                implementation deps.ktor.server.netty
                implementation deps.ktor.server.jwt
                implementation deps.ktor.server.locations
                implementation deps.ktor.server.jackson
                implementation deps.ktor.server.jackson_jsr310
                
                implementation deps.kotlinx.coroutines.jvm
            }
        }

        jvmTest {
            dependencies {
                implementation deps.kotlin.test.jvm
                implementation deps.kotlin.test.jvm_junit
            }
        }
    }
}

Folder Layout:

I'm starting the application using an IntelliJ Application configuration:

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/778

I have a section in config file:
app{
users{
user1 = some_value
user2 = some_value
user3 = some_value
}
}

How i cat get list properties from users section?
I can get properties by name, but I would like to get the entire list of properties in the section.
HoconApplicationConfig can use config.getConfig("app.users").entrySet(), but we dont can it (

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/672

Currently GAE support is broken. In spite of that we have a lot of unconfined in blocking servlet engine, there are several places where we use coroutine's default and IO dispatchers that is not possible in GAE.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1711

Ktor Version and Engine Used (client or server and name)
Server/netty 1.3.0

Describe the bug
When an URL for a defined Location is hit but some of the query parameters are missing HTTP404 is returned instead of HTTP400 which makes more sense. See example below.

To Reproduce
First we setup a simple application

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)

@Location("/user") data class User(val id: Long)

@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(Locations)
    routing {
        get<User> {
            call.respondText("User[id: ${id}]")
        }
    }
}

Now GETing localhost/user?id=123 works fine but GETing localhost/user returns HTTP 404 while HTTP 400 would certainly make more sense.

Expected behavior
400 status code when requesting a location with missing parameters.

Or at least a provided ability to override this behaviour manually.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1687

Ktor Version and Engine Used (client or server and name)
CachingHeaders 1.3.1

Describe the bug
The documentation for io.ktor.features.CachingHeaders.optionsProviders explicitly states "use first non null caching options". This is not true, and can be broken with a really simple setup.

To Reproduce
Steps to reproduce the behavior:

1. Run this app:

import io.ktor.application.call
import io.ktor.application.install
import io.ktor.features.CachingHeaders
import io.ktor.http.content.CachingOptions
import io.ktor.http.content.caching
import io.ktor.response.respondText
import io.ktor.routing.get
import io.ktor.routing.routing
import io.ktor.server.engine.embeddedServer
import io.ktor.server.netty.Netty
import java.time.ZonedDateTime

fun main() {
	embeddedServer(Netty, 8080) {
		install(CachingHeaders) {
			options { it.caching }
		}
		routing {
			get("/") {
				call.respondText("Hello") {
					caching = CachingOptions(
						expires = ZonedDateTime.now().plusSeconds(60)
					)
				}
			}
		}
	}.start(wait = true)
}

2. Invoke using browser/curl/etc.
3. Observe headers returned by the server

Expected behavior

HTTP/1.1 200 OK
Expires: Sat, 29 Feb 2020 12:49:30 GMT
Content-Length: 5
Content-Type: text/plain; charset=UTF-8
Connection: keep-alive

Actual behavior

HTTP/1.1 200 OK
Expires: Sat, 29 Feb 2020 12:49:30 GMT
Expires: Sat, 29 Feb 2020 12:49:30 GMT
Content-Length: 5
Content-Type: text/plain; charset=UTF-8
Connection: keep-alive

Notes

1. Putting more options { it.caching } into install will duplicate the header even more.

2. The documentation says:

   The basic feature is installed just like many others, but for it to do something, you have to define options blocks

   so it's very easy to end up with this, it should rather mention that the default behavior without any setup (install(CachingHeaders);) is:

   install(CachingHeaders) {
       options { it.caching }
   }
   

3. it looks to me that to satisfy "use first non null caching options" we need:

   fun optionsFor(content: OutgoingContent): CachingOptions? =
       optionsProviders.asSequence().mapNotNull { it(content) }.firstOrNull()
   

This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/68

When trying to use TestApplication and set any cookies it fails claiming feature is not configured while it clearly is... if you run the actual application it works fine, while all your tests will fail since you cannot set the cookies that need to pass around your session information.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1783

Say you have:

@Serializable
data class PagedData<T>(val data: T, val page: Int, val totalPages: Int, val pageSize: Int)

Where you know for sure that T is a class that has the annotations @Serializable or a list of a type that is annotated with @Serializable.

Why the serialization runtime is not able to infer what to use? It would be kind of pointless to explicitly build a serializer before every response as such.

Cross reference on kotlin.serialization issue board

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1556

Ktor Version and Engine Used (client or server and name)
version, client/server, additional ktor modules included
implementation 'io.ktor:ktor-server-netty:1.2.6'
implementation 'io.ktor:ktor-server-core:1.2.6'
implementation 'com.googlecode.ez-vcard:ez-vcard:0.10.5'

Describe the bug
Using Ktor to host a server, from client passing Vcard in multipart/form-data. Getting an exception saying "System.err: io.ktor.http.cio.ParserException: Character with code 59 is not allowed in header names, "

StackTrace::

01-10 16:06:40.925 9831 10021 W System.err: io.ktor.http.cio.ParserException: Character with code 59 is not allowed in header names,
01-10 16:06:40.925 9831 10021 W System.err: Content-Disposition: form-data; name="contacts|vcard246.vcf"Content-Transfer-Encoding: binaryContent-Type: multipart/form-data; charset=utf-8Content-Length: 156GIN:VCARDVERSION:3.0N;CHARSET=UTF-8: ;xxx.yyy.zzz.www; ; ;
01-10 16:06:40.926 9831 10021 W System.err: at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:275)
01-10 16:06:40.926 9831 10021 W System.err: at io.ktor.http.cio.HttpParserKt.parseHeaderNameFailed(HttpParser.kt:234)
01-10 16:06:40.926 9831 10021 W System.err: at io.ktor.http.cio.HttpParserKt.parseHeaderName(HttpParser.kt:220)
01-10 16:06:40.926 9831 10021 W System.err: at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:111)
01-10 16:06:40.926 9831 10021 W System.err: at io.ktor.http.cio.HttpParserKt$parseHeaders$2.invokeSuspend(Unknown Source:10)
01-10 16:06:40.926 9831 10021 W System.err: at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:171)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.ByteBufferChannel.resumeReadOp(ByteBufferChannel.kt:2211)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.ByteBufferChannel.flushImpl(ByteBufferChannel.kt:156)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.ByteBufferChannel.flush(ByteBufferChannel.kt:162)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.ByteBufferChannel.writeAsMuchAsPossible(ByteBufferChannel.kt:2719)
01-10 16:06:40.927 9831 10021 W System.err: at kotlinx.coroutines.io.ByteBufferChannel.writeFully(ByteBufferChannel.kt:1169)
01-10 16:06:40.927 9831 10021 W System.err: at io.ktor.server.netty.cio.RequestBodyHandler.copy(RequestBodyHandler.kt:133)
01-10 16:06:40.927 9831 10021 W System.err: at io.ktor.server.netty.cio.RequestBodyHandler.processContent(RequestBodyHandler.kt:95)
01-10 16:06:40.928 9831 10021 W System.err: at io.ktor.server.netty.cio.RequestBodyHandler$job$1.invokeSuspend(RequestBodyHandler.kt:38)
01-10 16:06:40.928 9831 10021 W System.err: at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
01-10 16:06:40.928 9831 10021 W System.err: at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:515)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
01-10 16:06:40.928 9831 10021 W System.err: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1007

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

netty

JVM Version, Operating System and Relevant Context

jdk 11, windows 10

Feedback

From my understanding the following code should redirect to the /login route, however nothing happens:

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)

@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(Authentication) {
        form {
            challenge = FormAuthChallenge.Redirect { "/login" }
        }
    }

    routing {
        get("/login") {
            call.respondText("LOGIN")
        }
        authenticate(optional = true) {
            static("/") {
                resources("static")
            }
        }

        authenticate {
            get("/") {
                call.respondText("HELLO WORLD!", contentType = ContentType.Text.Plain)
            }
        }
    }
}

The problem seems to be that static (static("/")) has the same root path as get ( get("/")) in conjunction with authenticate(optional = true) .

The redirect works again, if the paths are different or/and optional = false.

If I activate call logging I get: INFO Application - Unhandled: GET - /

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/124

It would be great to be able to configure logging directly in the application.conf instead of maintaining a separate logback.xml configuration file

For example, Dropwizard offers this.

It allows therefore to configure all aspects of application configuration from one config file.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1012

ktor-server-netty 1.1.3

The following code makes it impossible to use GET requests with a content payload:
https://github.com/ktorio/ktor/blob/d4e1a7caaa031fb466beb1cc102192cc88623bfe/ktor-server/ktor-server-netty/jvm/src/io/ktor/server/netty/http1/NettyHttp1Handler.kt#L46-L48

The HTTP specification does not limit content payloads to specific HTTP methods, so GET request can have one as well.

Many HTTP clients support payloads for GET requests and Ktor should not make the assumption that GET request never contain one.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1072

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

ItelliJ - Gradle, Google Chrome

JVM Version, Operating System and Relevant Context

1.8 JVM, Windows 10

Feedback

After running the the "Hello World" Quick Start inside my Server, the server starts up fine and I receive the following output with no errors:

[12:06:11.215 INFO ] application          | No ktor.deployment.watch patterns specified, automatic reload is not active
[12:06:11.340 INFO ] application          | Responding at http://0.0.0.0:8080

When I head over to http://127.0.0.1:8080/demo I receive ERR_EMPTY_RESPONSE and error code 52 with a CURL request. I get no errors at all other than the webpage returning no response.

My full code looks like this:

override fun bindNet(server: Server, world: World) {
        val server = embeddedServer(Netty, port = 8080) {
            routing {
                get("/") {
                    call.respondText("Hello World!", ContentType.Text.Plain)
                }
                get("/demo") {
                    call.respondText("HELLO WORLD!")
                }
            }
        }
        server.start(wait = true)
    }

I am using Netty to bind other ports in my application which may be conflicting on the IP, I have tried to change the port to a different port but no success and also add the local address 127.0.0.1 to the host with no success.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1909

Subsystem
ktor-server-netty

Is your feature request related to a problem? Please describe.
I'm trying to add some GlobalTrafficShapingHandler to ktor, but I cannot do anything because Ktor overwrites what I am trying to do in the configureBootstrap lambda. I cannot set the childHandler or the channelFactory , or even influence what they do in any way. I cannot touch childHandler nor channelFactory (it's set via the channel() call).

ServerBootstrap().apply {
	configuration.configureBootstrap(this)
	group(connectionEventGroup, workerEventGroup)
	channel(connectionEventGroup.channel.java)
	childHandler(
		NettyChannelInitializer(
			pipeline, environment,
			callEventGroup, engineDispatcherWithShutdown, dispatcherWithShutdown,
			connector,
			configuration.requestQueueLimit,
			configuration.runningLimit,
			configuration.responseWriteTimeoutSeconds,
			configuration.requestReadTimeoutSeconds,
			configuration.httpServerCodec
		)
	)
}

Describe the solution you'd like
I'd want a way to expose the underlying NettyPipeline, or at least have some way to meaninfully use the configureBootstrap hook. I would also take a hook called in override fun initChannel(ch: SocketChannel) { as that would let me do the configuration I want.

Motivation to include to ktor
Ktor already takes effort to expose this hook, but I've found the things I really want to do with this hook aren't allowed.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1941

Ktor Version and Engine Used (client or server and name)
1.3.2, Netty server, FreeMarker and StatusPages

Describe the bug
If an exception is thrown while generating a html page using FreeMarker (ie. FreeMarker throws a TemplateException), the StatusPages feature does not consume it. I've tried catching Throwable and TemplateException:

exception<TemplateException> {
    Rollbar.error("Error generating page: ", it)
    call.respond(FreeMarkerContent(
        "error.ftl",
        mapOf("data" to ErrorData(
            500,
            "Oops! An error occurred while we were generating that page!",
            "Hit the back button on your browser and try again."
        ))
    ))
}

exception<Throwable> {
    Rollbar.error(null, it)
    call.respond(FreeMarkerContent(
        "error.ftl",
        mapOf("data" to ErrorData(
            500,
            "Oops! Something went wrong on our end!",
            it.localizedMessage
        ))
    ))
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1962

Ktor Version and Engine Used (client or server and name)
Server CIO, 1.3.2

Describe the bug
If we don't call call.respond the server freezes and holds the connection with CIO server(without any message)
The Netty treats this sample as correct response and respond with status code Unauthorized

To Reproduce
Make the request with CIO server:

get("unauthorized") {
    call.response.status(HttpStatusCode.Unauthorized)
    call.response.header(HttpHeaders.WWWAuthenticate, "Basic realm=\"TestServer\", charset=UTF-8")
}

Expected behavior
Respond with 5xx Internal server error or a correct response in that case

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/336

I runned the hello world example and it's ok. When I launch a second instance at the same time, I get java.net.BindException: Address already in use: bind. Perfect. But I expected the second instance to end with a failure. Instead I get a hanging netty application (a thread waiting in some netty routine).

Here is the code: https://github.com/jarekczek/ktorSimple/blob/master/src/main/java/WwwServer.kt
It never gets after the server.start(wait = false) line.

I know a workaround. Catch the exception and System.exit().

What's also strange is that if I don't do server.stop, the program never ends. I expect that when my application comes to the end of the main method, the program would stop. It works this way in finnagle. But not here.

So a final question: should netty prevent the application from terminating?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1887

Ktor Version and Engine Used (client or server and name)
1.3.2, server/client, CIO engine

Describe the bug
I don't see uninstallFeature function, un-intercepting the pipeline.
Tested with DefaultHeaders and CORS.

To Reproduce
Server code,

fun main() {
    embeddedServer(CIO, host = "localhost", port = 8080) {    
        install(CORS) {
            host("AHost.com")
        }
        
        install(DefaultHeaders)
        
        uninstallAllFeatures() // This is where uninstallFeature will be called
        
        routing {  
            get { 
                call.respond("Hello")
            }
        }
    }.start(wait = true)
}

A client code (typically a simple get request),

fun main() = runBlocking {
    val client = HttpClient(CIO)
    
    client.get<HttpResponse>(port = 8080) {
        
        header(HttpHeaders.Accept, ContentType.Application.Json)
        header(HttpHeaders.Origin, "http://AHost.com")
        
    }.run {
        
        println(headers)
        
    }
    
}

Result,

Headers [Vary=[Origin], Access-Control-Allow-Origin=[http://AHost.com], Date=[Fri, 22 May 2020 00:21:53 GMT], Server=[ktor-server-core/1.3.2 ktor-server-core/1.3.2], Content-Length=[5], Content-Type=[text/plain; charset=UTF-8]]

Expected behavior
Features actually be uninstalled. Meaning they won't intercept the pipeline anymore.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1656

Ktor Version and Engine Used (client or server and name)
1.3.0

Describe the bug
CORS can't pass on some none standard orgin like chrome-extension://mfgdmpfihlmdekaclngibpjhdebndhdj

To Reproduce
open some new page in chrome or other browser,just call any api,CORS throw would error(brower side).

Expected behavior
should be AnyOrigin pass by CORS, when we config it as host("*") or anyHost()

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1959

CIO Server ktor-server-cio:1.3.2-1.4-M2

Describe the bug
CIO server always start on "0.0.0.0" - does not respect "connector" configuration

To Reproduce
Steps to reproduce the behavior:

embeddedServer(CIO, applicationEngineEnvironment {
            module { ... }
            connector {
                host = "192.168.0.1"
                port = 8080
            }
})

Server will start on "0.0.0.0:8080" instead of "192.168.0.1:8080"

Expected behavior
Server respect connector configuration.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1868

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2

Describe the bug
Missing header name should fail(see https://tools.ietf.org/html/rfc2616#section-4.2)

To Reproduce
Steps to reproduce the behavior:
Run code:

import io.ktor.http.cio.parseRequest
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking

fun main() {
    test(
        """
        GET / HTTP/1.1
        Host: www.example.com
        : value


        """.trimIndent()
    )
}

fun test(http: String) {
    val request = runBlocking { parseRequest(ByteReadChannel(http)) }!!
    println("method=${request.method};version=${request.version};uri={${request.uri}};headers=[${request.headers}]")
}

Expected behavior
An error expected - exception or null result.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1882

Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2

Describe the bug
Status-code must be 3-digit(see https://tools.ietf.org/html/rfc2616#section-6.1.1)

To Reproduce
Steps to reproduce the behavior:
Run code:

import io.ktor.http.cio.parseResponse
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking


fun main() {
    listOf(
        """
        HTTP/1.1 0 OK


        """.trimIndent(),
        """
        HTTP/1.1 10 OK


        """.trimIndent(),
        """
        HTTP/1.1 1000 OK


        """.trimIndent(),
        """
        HTTP/1.1 10000 OK


        """.trimIndent()
    ).forEach { test(it) }

}

fun test(http: String) {
    val response = runBlocking { parseResponse(ByteReadChannel(http)) }!!
    println("status=${response.status};statusText={${response.statusText}};headers=[${response.headers}];version=${response.version}")
}

Expected behavior
An error expected - exception or null result.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/129

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1885

Ktor Version and Engine Used (client or server and name)
1.3.2, Netty server, routes

ktor_version=1.3.2, kotlin_version=1.3.70

implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version")
    implementation("io.ktor:ktor-server-netty:$ktor_version")
    implementation("ch.qos.logback:logback-classic:$logback_version")
    testImplementation("io.ktor:ktor-server-tests:$ktor_version")

Describe the bug
I migrated one of my project form php to ktor and deployed it in production for about 5k users. After one day server stopped working and linux file opened limits exceed which was 20k and when i checked with command ss -ant | grep :8383 | wc -l connections were 20k+ but i don't have users more than 5k which were not all active. Then i switched my floating ip to old php server and no more request to this server next day when i checked again with command ss -ant | grep :8383 | wc -l connections were still 20k+ even no more new request were coming and all connections were in ESTAB state.

So after that i created new project which was empty with ktor, kotlin dsl i mentioned its dependencies above and run that on that server Then i run this command watch -n 1 'ss -ant | grep {MY_IP}' on server to filter connections for my ip first it was no connection from my ip on port 8383 then i send single request from postman on SERVER_IP:8388 which was simple get request and response was from call.respond("Testing") and on server terminal it show me connection on 8383 with my ip. In postman request was complete and i got result. Then i closed postman and keep checking when it will close my connection from server. I checked that after an hour still connection was not closed and when i requested again it was still using that same connection. So it was reusing connection and not closing even after some hours. I'm not sure but may be error with keep alive or http2. I'm not sending any header i request.

I didn't tried any other ktor or kotlin version and I'm not able to reproduce it local.
I'm running jar file generated with shadowjar on linux server with systemd service.

To Reproduce

1. create new project with ktor version 1.3.2 and kotlin 1.3.70 with GradleKotlinDsl

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(Routing) {
        get("/") {
            call.respond("Testing")
        }
    }
}

2. create jar file with shadowjar

tasks {
    named<ShadowJar>("shadowJar") {
        archiveBaseName.set("shadow")
        mergeServiceFiles()
        manifest {
            attributes(mapOf("Main-Class" to mainClasses))
        }
        transform(com.github.jengelman.gradle.plugins.shadow.transformers.ServiceFileTransformer::class.java) {
            setPath("META-INF/services")
            include("org.eclipse.jetty.http.HttpFieldPreEncoder")
        }
        archiveFileName.set("example.jar")
    }
}

3. upload on ubuntu server and create its systemd service and run with systemctl start example
   service file:

[Unit]
Description=example
After=syslog.target

[Service]
User=root
LimitNOFILE=20536
ExecStart=/var/www/example/example-script
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

example-script:

#!/bin/sh
cd /var/www/example
java -jar exmaple.jar

Expected behavior
When user make request to http://SERVER_IP:APP_PORT it should close it after request complete but it keep opened it until we will stop server. Checking with ss -ant | grep {CLIENT_PUBLIC_IP}

Screenshots
This is screenshot which i get after an hour when made last request in screenshot last connection with port 8383 that is from my machine and it is still not closed.

Kotlin version: 1.3.70
Ktor version: 1.3.2

In situations when receiving large cookies and the headers exceed the 8K limit (using Netty as engine), not all headers get loaded.

Example 1:

• When we have the ContentNegotiation feature enabled
• And a content-type: application/json is present on the request
• And the cookie header is very large (the sum of all headers exceeding 8K)
• Then the ContentNegotiation feature resolves the content type to Any (*/*) - probably because it can't load all headers
• This results in a situation where the server responds with 415 - Unsupported Media Type.

In this particular example probably a 431 - Requests header fields too large is more appropriate.

Example 2:

• Consider the hypothesis from Example 1
• When the content-type: application/json header is loaded before the cookie header
• Then the request passes successfully through the ContentNegotiation feature
• But the request hangs

This seems like an issue where Ktor does not correctly interpret errors from the Netty engine.

Sample engine error:

DefaultHttpRequest(decodeResult: failure(io.netty.handler.codec.TooLongFrameException: HTTP header is larger than 8192 bytes.), version: HTTP/1.1)
POST /v1/api HTTP/1.1
Host: localhost:8080
Accept: */*
cache-control: no-cache
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
content-type: text/plain;charset=UTF-8

We recently got some alerts in our webserver because ktor threw an uncaught exception. Ideally these are handled and a 400 is sent back.

Here is the exception:
Wrong HEX escape: %uf, in /%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/winnt/win.ini

Stacktrace:

at io.ktor.http.CodecsKt.decodeImpl (Codecs.kt:197)
at io.ktor.http.CodecsKt.decodeScan (Codecs.kt:146)
at io.ktor.http.CodecsKt.decodeURLPart (Codecs.kt:140)
at io.ktor.http.CodecsKt.decodeURLPart$default (Codecs.kt:139)
at io.ktor.routing.RoutingResolveContext.parse (RoutingResolve.kt:70)
at io.ktor.routing.RoutingResolveContext.<init> (RoutingResolve.kt:49)
at io.ktor.routing.Routing.interceptor (Routing.kt:31)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend (Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke (Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend (ContentNegotiation.kt:107)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke (ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend (DefaultEnginePipeline.kt:120)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke (DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend (NettyApplicationCallHandler.kt:40)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke (NettyApplicationCallHandler.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched (Undispatched.kt:55)
at kotlinx.coroutines.CoroutineStart.invoke (CoroutineStart.kt:111)
at kotlinx.coroutines.AbstractCoroutine.start (AbstractCoroutine.kt:158)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch (Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch (Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest (NettyApplicationCallHandler.kt:30)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead (NettyApplicationCallHandler.kt:24)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600 (AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run (AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1767

Subsystem
Server

Is your feature request related to a problem? Please describe.
We want to use client-side certificates to authenticate HTTPS requests from connected devices. In our code, we need to get the client certificate information to determine the DN in the certificate.

In a regular servlet, client certificate information can be accessed using request.getAttribute("javax.servlet.request.X509Certificate"), which returns the certificate chain.

Our strategy to get this to work in ktor was to run our application as a WAR in a Tomcat instance configured to do client authentication. Unfortunately, request attributes do not seem to be available. (Note that a Spring Boot application deployed under the same Tomcat can access these attributes, so we know Tomcat is configured correctly for TLS.)

Describe the solution you'd like
We'd like some way to access servlet request attributes from ktor application code.

Motivation to include to ktor
Client certificates are an important part of IoT deployments, where ktor would shine because typically a server needs to manage a lot of concurrent connections. Ktor itself doesn't support client certificate authentication - our solution would be a workaround if only we could access request attributes.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/544

At the moment Ktor is trying to compress even the steel.
I added this method in my core lib to overcome that.
I think there should be a similar thing active by default. What do you reckon?

fun Application.installDefaultCompresion() {
    install(Compression) {
        gzip {
            matchContentType(
                ContentType.Text.Any,
                ContentType.Application.JavaScript,
                ContentType.Application.Json,
                ContentType.Application.Rss,
                ContentType.Application.Xml,
                ContentType.Application.Xml_Dtd,
                ContentType.Application.Atom,
                ContentType.Image.SVG,
                ContentType.Image.XIcon
            )

            minimumSize(1400) // not worth compressing
        }
    }
}

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/557

Using the install for CachingHeaders I want to configure it in such a way it produces this header:

Cache-Control: no-cache, no-store, must-revalidate

I cannot do it.
For example I tried this:

    options { outgoingContent ->
        when (outgoingContent.contentType?.withoutParameters()) {
            ContentType.Text.CSS -> CachingOptions(CacheControl.NoCache(null))
            else -> null
        }
    }
    options { outgoingContent ->
        when (outgoingContent.contentType?.withoutParameters()) {
            ContentType.Text.CSS -> CachingOptions(CacheControl.NoStore(null))
            else -> null
        }
    }

it makes it twice:

cache-control: no-cache
cache-control: no-store

The best would be if options can return a lamda with a list of CachingOptions as return value, then the system would add all of them. Or allow caching options to accept varargs?

Also, why is "must revalidate" tied to max age?
See this, it can be also on it's own:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#Examples

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/82

If exception happens in ktor and is not handled by an interceptor such as StatusPages, it should propagate up to host and be handled there without using any ktor facilities like response pipeline. The reason is that we don't know which part of the system were at fault and it could very well be response pipeline itself. So unhandled exception should be propagated to the host-specific facilities and responded with 500 status code natively there.

Note, that it would make 500 Internal Server Error status produced in this case not interceptable by StatusPage feature on a per code basis, but exception filter in the same feature can be installed on Throwable and do pretty much any custom handling there. At this moment exception would be considered handled and won't be part of this issue.

Invoke WebSocketSession.close().
It causes the following exception:

kotlinx.coroutines.experimental.channels.ClosedSendChannelException: Channel was closed
	at kotlinx.coroutines.experimental.channels.Closed.getSendException(AbstractChannel.kt:1008)
	at kotlinx.coroutines.experimental.channels.AbstractSendChannel.offer(AbstractChannel.kt:185)
	at kotlinx.coroutines.experimental.channels.AbstractSendChannel.send(AbstractChannel.kt:175)
	at kotlinx.coroutines.experimental.channels.ChannelCoroutine.send$suspendImpl(ChannelCoroutine.kt:33)
	at kotlinx.coroutines.experimental.channels.ChannelCoroutine.send(ChannelCoroutine.kt)
	at kotlinx.coroutines.experimental.channels.LazyActorCoroutine.send(Actor.kt:192)
	at io.ktor.http.cio.websocket.WebSocketWriter.flush(WebSocketWriter.kt:122)
	at io.ktor.http.cio.websocket.RawWebSocket.flush(RawWebSocket.kt:37)
	at io.ktor.websocket.DelegatedWebSocketServerSession.flush(WebSocketServerSession.kt)
	at io.ktor.http.cio.websocket.DefaultWebSocketSessionImpl.flush(DefaultWebSocketSessionImpl.kt)
	at io.ktor.websocket.DelegatedDefaultWebSocketServerSession.flush(WebSocketServerSession.kt)
	at io.ktor.http.cio.websocket.WebSocketSessionKt.close(WebSocketSession.kt:58)

WebSocketSession.close invokes flush that fails because the websocket is already closed at the moment.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1929

Ktor Version and Engine Used (client or server and name)
ktor-cio server 1.3.2

Describe the bug
I am getting an internal error when force-closing server with opened web-socket connection
Here is the stack trace:

13:32:20.839 [DefaultDispatcher-worker-4] ERROR io.ktor.server.cio.HttpServer - Unhandled exception caught for CoroutineName(http-pipeline-writer)
kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[ExperimentalCoroutineDispatcher@45e6d6a4[scheduler = DefaultDispatcher@6341301e[Pool Size {core = 10, max = 1024}, Worker States {CPU = 6, blocking = 0, parked = 8, dormant = 0, terminated = 0}, running workers queues = [0c, 0c, 0c, 0c, 0c, 0c], global CPU queue size = 5, global blocking queue size = 0, Control State {created workers= 14, blocking tasks = 0, CPUs acquired = 6}]], Continuation at io.ktor.server.cio.backend.ServerPipelineKt$startServerConnectionPipeline$1$outputsActor$1.invokeSuspend(ServerPipeline.kt:49)@2482cf33]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	... 4 common frames omitted

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1158

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Jetty server

JVM Version, Operating System and Relevant Context

JDK 11, Linux 5.0

Feedback

Given the following code:

import io.ktor.application.call
import io.ktor.response.respondText
import io.ktor.routing.get
import io.ktor.routing.routing
import io.ktor.server.engine.embeddedServer
import io.ktor.server.jetty.Jetty

fun main() {
  val server = embeddedServer(Jetty, port = 9010) {
    routing {
      get("/foo:{bar}/{baz}") {
        call.respondText("Received route1 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
      }
      get("/{bar}/{baz}") {
        call.respondText("Received route2 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
      }
    }
  }
  server.start(wait = true)
}

The first route has a static component beginning with /foo: whereas the second route is all dynamic parameters. The first route is therefore of higher quality for matching purposes.

With this call:

http get :9010/foo:a/b

the result is as expected: Received route1 bar=a, baz=b

However, when changing the code to wrap the second route in an authenticate block like this:

    routing {
      get("/foo:{bar}/{baz}") {
        call.respondText("Received route1 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
      }
      authenticate {
        get("/{bar}/{baz}") {
          call.respondText("Received route2 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
        }
      }
    }

I would expect the routing decision to be the same, but:

http get :9010/foo:a/b

now attempts to execute the lower quality authenticated path:

HTTP/1.1 401 Unauthorized

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/445

Hello,
I have a problem when I do : call.sessions.get<MySession>() with a wrong header. I have the following error 2018-06-18 15:59:07.681 [nettyCallPool-4-1] DEBUG Application - Failed to lookup session: java.util.NoSuchElementException: No session data found for id 684eb3846f0441113ab51e24471b8062f1d30f5290a11046d17d5725470fdce7.

"For unexistant sessions that are not available, would be like if no session was provided and you can do whatever you need in your route handler"

Thank for your help !

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/482

Problem description: When receiving an empty multipart-form request on a POST route, the request handler throws an IOException, preventing graceful handling or the display of an error message. Instead, a 500 error message is returned.

Minimum working example:

import io.ktor.application.*
import io.ktor.content.readAllParts
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.post
import io.ktor.routing.routing

fun main(args: Array<String>): Unit = io.ktor.server.netty.DevelopmentEngine.main(args)

fun Application.module() {
    routing {
        post("/add") {
            if(call.request.isMultipart()) {
                val multipart = call.receiveMultipart()
                val formItems = multipart.readAllParts() // <-- this breaks when sent an empty request
                call.respondText("recvd: $multipart")
            }
            else {
                call.respond("no multipart, fix request!")
            }
        }
    }
}

Send a request with an empty multipart body to see the issue:

curl -X POST \
  http://localhost:8080/add \
  -H 'Cache-Control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'

Error Message:

22:57:11.866 [nettyCallPool-4-4] ERROR Application - Unhandled: POST - /add
java.io.IOException: Broken delimiter occurred
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.doResume(Delimited.kt:57)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt:1746)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiterSuspend(Delimited.kt:55)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiter(Delimited.kt:50)
	at io.ktor.http.cio.MultipartKt.boundary(Multipart.kt:89)
	at io.ktor.http.cio.MultipartKt$parseMultipart$1.doResume(Multipart.kt:158)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
	at kotlinx.coroutines.experimental.DispatchedKt.resumeCancellable(Dispatched.kt:209)
	at kotlinx.coroutines.experimental.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:35)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce(Produce.kt:95)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce$default(Produce.kt:88)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:145)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:138)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:33)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:31)
	at io.ktor.server.engine.DefaultTransformKt.multiPartData(DefaultTransform.kt:70)
	at io.ktor.server.engine.DefaultTransformKt.access$multiPartData(DefaultTransform.kt:1)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.doResume(DefaultTransform.kt:33)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:64)
	at io.sebi.ApplicationKt$module$1$1.doResume(application.kt:31)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.routing.Routing.executeResult(Routing.kt:100)
	at io.ktor.routing.Routing.interceptor(Routing.kt:25)
	at io.ktor.routing.Routing$Feature$install$1.doResume(Routing.kt:66)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt:51)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.doResume(DefaultEnginePipeline.kt:66)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.doResume(NettyApplicationCallHandler.kt:31)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt:10)
	at kotlinx.coroutines.experimental.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:44)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:113)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch(Builders.common.kt:72)
	at kotlinx.coroutines.experimental.BuildersKt.launch(Unknown Source)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:64)
	at kotlinx.coroutines.experimental.BuildersKt.launch$default(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:844)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1074

Ktor Version

1.1.3 & 1.1.4

Ktor Engine Used (client or server and name)

The issue is only about a server.
Netty, Jetty, Tomcat, CIO.

JVM Version, Operating System, and Relevant Context

java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)

ProductName:    Mac OS X
ProductVersion: 10.14.4
BuildVersion:   18E226

Feedback

There's (maybe) a bug:

Steps to reproduce

The simple code below is enough to reproduce.

webSocket("/ws") {
    close()
    // or terminate()
    // or close(CloseReason(...))
    // or close(Exception())
}

Connection is not closing with Netty or Jetty engine, but closing with CIO or Tomcat.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1802

Hi, I'm new to ktor and would kindly ask you to support me if possible.

package io.dodalovic.routes

import io.dodalovic.API_VERSION
import io.dodalovic.auth.JwtService
import io.dodalovic.auth.MySession
import io.dodalovic.repo.Repository
import io.ktor.application.application
import io.ktor.application.call
import io.ktor.application.log
import io.ktor.http.HttpStatusCode
import io.ktor.http.Parameters
import io.ktor.locations.KtorExperimentalLocationsAPI
import io.ktor.locations.Location
import io.ktor.locations.post
import io.ktor.request.receive
import io.ktor.request.receiveParameters
import io.ktor.response.respond
import io.ktor.response.respondText
import io.ktor.routing.Route
import io.ktor.sessions.sessions
import io.ktor.sessions.set

const val USERS = "$API_VERSION/users"
const val USER_LOGIN = "$USERS/login"
const val USER_CREATE = "$USERS/create"

@KtorExperimentalLocationsAPI
@Location(USER_LOGIN)
class UserLoginRoute

@KtorExperimentalLocationsAPI
@Location(USER_CREATE)
class UserCreateRoute

@KtorExperimentalLocationsAPI
fun Route.users(
    db: Repository,
    jwtService: JwtService,
    hashFunction: (String) -> String
) {
    post<UserCreateRoute> {
        val signupParameters = call.receive<Parameters>()
        val password = signupParameters["password"]
            ?: return@post call.respond(
                HttpStatusCode.Unauthorized, "Missing Fields"
            )
        val displayName = signupParameters["displayName"]
            ?: return@post call.respond(
                HttpStatusCode.Unauthorized, "Missing Fields"
            )
        val email = signupParameters["email"]
            ?: return@post call.respond(
                HttpStatusCode.Unauthorized, "Missing Fields"
            )
        val hash = hashFunction(password)
        try {
            val newUser = db.addUser(email, displayName, hash)
            newUser?.userId?.let {
                call.sessions.set(MySession(it))
                call.respondText(
                    jwtService.generateToken(newUser),
                    status = HttpStatusCode.Created
                )
            }
        } catch (e: Throwable) {
            application.log.error("Failed to register user", e)
            call.respond(HttpStatusCode.BadRequest, "Problems creating User")
        }
    }
}

I'm getting:

2020-04-17 22:12:48.991 [nioEventLoopGroup-4-4] ERROR Application - Unhandled: POST - /v1/users/create
java.lang.RuntimeException: Unable to invoke no-args constructor for interface io.ktor.http.Parameters. Registering an InstanceCreator with Gson for this type may fix this problem.
	at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:228)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:212)
	at com.google.gson.Gson.fromJson(Gson.java:932)
	at com.google.gson.Gson.fromJson(Gson.java:870)
	at io.ktor.gson.GsonConverter.convertForReceive(GsonSupport.kt:43)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:169)
	at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:110)
	at io.dodalovic.routes.UserRouteKt$users$1.invokeSuspend(UserRoute.kt:74)

when the line:

val signupParameters = call.receive<Parameters>()

gets called. Are there any prerequisites?

Many thanks!

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1241

Currently, if you try to match
http://localhost:8080/http://foo.com/image.url

Then both {...} and * fail. The {...} will capture but by changing the path to http:foo.com/image.url

This kind of path matching is very useful for endpoints that accept other urls.

I could use a query parameter but I am attempting to write a drop in replacement for an existing service.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1726

Subsystem
Server - Serialization

Is your feature request related to a problem? Please describe.
Just like in #875 it would be great if a KType would be passed to ContentConverter.convertForSend since it'd be impossible to correctly serialize parameterized types otherwise.

Describe the solution you'd like
Not sure how this could be done in a binary compatible way. I suppose changing the context's subject would be binary compatible since it's a generic but it would be a source incompatible change unless the subject is also declared as out.

Motivation to include to ktor
See the examples in #875. Writing polymorphic types or parameterized types can cause issues unless the expected type is know in advance.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1387

Ktor Version and Engine Used (client or server and name)
1.2.5, server, ktor-server-cio

Describe the bug

my application.conf:

ktor {
    application {
        modules = [ adeln.MainKt.module ]
    }
    deployment {
        host = "127.0.0.1"
        port = 8080
    }
}

When I run netstat -plnt:

With ktor-server-cio deployment.host is ignored and my server is accessible externally:

tcp6       0      0 :::8080                 :::*                    LISTEN      24413/java          

To Reproduce

1. Use my application conf
2. Launch the server using ktor-server-cio
3. Run netstat -plnt and see that deployment.host is ignored

Expected behavior
With ktor-server-netty deployment.host is applied correctly:

tcp6       0      0 127.0.0.1:8080          :::*                    LISTEN      24978/java          

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/268

Under certain circumstances it happens that Ktor attempts to send a double response on a HTTP request.

A sample repository with an MRC: https://github.com/noncom/ktor-double-response-issue

Steps to reproduce: the KtorTest main object contains 3 samples of the routing structure. Comment/uncomment the samples to get the behavior described in the corresponding comments.

The issue: upon the attempt to access the localhost:8080 address with the first routing sample being active, Ktor gives the following error:

13:45:29.158 [nettyCallPool-4-1] ERROR ktor.application - 200 OK: GET - /
io.ktor.server.engine.BaseApplicationResponse$ResponseAlreadySentException: Response has already been sent
	at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.doResume(BaseApplicationResponse.kt:30)
	at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.invoke(BaseApplicationResponse.kt)
	at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.invoke(BaseApplicationResponse.kt:14)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:31)
	at io.ktor.pipeline.PipelineContext.proceedWith(PipelineContext.kt:20)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.doResume(DefaultTransform.kt:15)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:31)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:16)
	at io.ktor.content.StaticContentKt$defaultResource$1.doResume(StaticContent.kt:150)
	at io.ktor.content.StaticContentKt$defaultResource$1.invoke(StaticContent.kt)
	at io.ktor.content.StaticContentKt$defaultResource$1.invoke(StaticContent.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:31)
	at io.ktor.pipeline.PipelineContext$proceed$1.doResume(PipelineContext.kt)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:54)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:53)
	at kotlinx.coroutines.experimental.DispatchTask.run(CoroutineDispatcher.kt:123)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

What is expected: Ktor should not attempt to send more than one response per request.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1965

Ktor Version and Engine Used (client or server and name)

ktor_version=1.3.2
kotlin_version=1.3.70

Describe the bug

When deserializing, io.ktor.utils.io.ByteReadChannelKt.readUTF8Line() is called and that tries to reads one char too far unless the string returned from MySessionSerializer<T>.serialize() has a trailing newline.

To Reproduce

class SerializableSessionSerializer<T>() : SessionSerializer<T> {
    override fun serialize(session: T): String {
        val bo = ByteArrayOutputStream()
        val so = ObjectOutputStream(bo)
        so.writeObject(session)
        so.flush()
        val enc = Base64.getEncoder().encodeToString(bo.toByteArray())
        // return enc + "\n"
        return enc
    }

    override fun deserialize(text: String): T {
        val b: ByteArray = Base64.getDecoder().decode(text)
        val bi = ByteArrayInputStream(b)
        val si = ObjectInputStream(bi)
        return si.readObject() as T
    }
}
...
    install(Sessions) {
        cookie<MySessionData>("mycookie", storage = SessionStorageMemory()) {
            serializer = SerializableSessionSerializer<MySessionData>()
        }
    }

Expected behavior

I expected that any serialized string would've been fine. No documentation to the contrary: Serializers - Servers - Ktor

Stack trace
As seen in the console:

2020-06-23 00:48:34.721 [nioEventLoopGroup-4-1] ERROR Application - Unhandled: GET - /
java.lang.IllegalArgumentException: newPosition > limit: (9105 > 9104)
	at java.base/java.nio.Buffer.createPositionException(Buffer.java:318)
	at java.base/java.nio.Buffer.position(Buffer.java:293)
	at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1086)
	at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:262)
	at io.ktor.utils.io.charsets.UTFKt.decodeUTF8Line_array(UTF.kt:661)
	at io.ktor.utils.io.charsets.UTFKt.decodeUTF8Line(UTF.kt:36)
	at io.ktor.utils.io.ByteBufferChannel$readUTF8LineToUtf8Suspend$2.invokeSuspend(ByteBufferChannel.kt:2120)
	at io.ktor.utils.io.ByteBufferChannel$readUTF8LineToUtf8Suspend$2.invoke(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteBufferChannel.lookAheadSuspend$suspendImpl(ByteBufferChannel.kt:1877)
	at io.ktor.utils.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8LineToUtf8Suspend(ByteBufferChannel.kt:2116)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8LineToAscii(ByteBufferChannel.kt:2056)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8LineTo$suspendImpl(ByteBufferChannel.kt:2144)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8LineTo(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8Line$suspendImpl(ByteBufferChannel.kt:2148)
	at io.ktor.utils.io.ByteBufferChannel.readUTF8Line(ByteBufferChannel.kt)
	at io.ktor.utils.io.ByteReadChannelKt.readUTF8Line(ByteReadChannel.kt:223)
	at io.ktor.sessions.SessionTrackerById$load$2.invokeSuspend(SessionTrackerById.kt:35)
	at io.ktor.sessions.SessionTrackerById$load$2.invoke(SessionTrackerById.kt)
	at io.ktor.sessions.SessionStorageMemory.read(SessionStorageMemory.kt:34)
	at io.ktor.sessions.SessionTrackerById.load(SessionTrackerById.kt:34)
	at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:205)
	at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:59)
	at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:120)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:40)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
	at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:158)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
	at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:30)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:24)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:59)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:368)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:835)

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/831

Ktor Version

1.0.1

Ktor Engine Used(client or server and name)

Jetty Server

JVM Version, Operating System and Relevant Context

1.8.0 Update 191, macOS 10.14.2

Feedback

I use post<List<ConfigCorepage>>("/api/pageName/update") to accept the json content from user request, but ktor couldn't recognize the type List<ConfigCorepage>, it use List instead.

I debug the code, and find that the code here shouldn't use T::class.

Jackson use a TypeReference class for preserving class info, ktor should use the same tricks.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/939

According to MDN and W3 the two headers Accept and Accept-Language are simple headers and allowed by default or CORS requests, so they don't need be specified in Access-Control-Request-Headers.

Unfortunately Internet Explorer 11 specifies Accept anyway and because Ktor doesn't list them as default headers CORS requests from IE 11 get rejected by 403 Forbidden:

https://github.com/ktorio/ktor/blob/cf7d8e43c80a4adfe3363a0191cc5df9afb78bdc/ktor-server/ktor-server-core/jvm/src/io/ktor/features/CORS.kt#L225-L234

Access-Control-Request-Headers is checked in the following code and .CorsDefaultHeaders being used through .allHeadersSet/.allHeaders:

https://github.com/ktorio/ktor/blob/cf7d8e43c80a4adfe3363a0191cc5df9afb78bdc/ktor-server/ktor-server-core/jvm/src/io/ktor/features/CORS.kt#L153-L157

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1293

Ktor client version: 1.2.3

URLBuilder(host = "localhost").apply {
    path("?")
    parameters["?"] = "?"
}
    .buildString()
    .also(::println)

will produce:

http://localhost/??%3F=%3F

I would expect the following url:

http://localhost/%3F?%3F=%3F

URLBuilder uses encodeURLQueryComponent function to encode a path: https://github.com/ktorio/ktor/blob/1.2.3/ktor-http/common/src/io/ktor/http/URLBuilder.kt#L50
Possibly it should use encodeURLPath?

Not sure it is a bug, but a feature. Fixing it will break backward compatibility.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1906

Ktor Version and Engine Used
Ktor 1.3.2, Netty

Here is the failing test:

@Test
fun testRequestWithNonSimpleContentTypeShouldFail() {
    withTestApplication {
        application.install(CORS) {
            anyHost()
        }
        application.routing {
            post("/") {
                call.respond(HttpStatusCode.OK)
            }
        }

        handleRequest(HttpMethod.Post, "/") {
            addHeader(HttpHeaders.Origin, "http://my-host")
            addHeader(HttpHeaders.ContentType, "application/json")
            setBody("""{"cors":"test"}""")
        }.apply {
            assertEquals(HttpStatusCode.Forbidden, response.status())
        }
    }
}

I've read it that CORS feature doesn’t allow non-simple request body content types by default but this test fails. Am I understanding it wrong?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1869

Ktor Version and Engine Used (client or server and name)
1.3.2, server

Describe the bug
ApplicationCall.request.origin.remoteHost holds the scheme instead of the version (like 'http' instead of HTTP/1.1).
The root cause is this line.

I provided a fix here: https://github.com/ktorio/ktor/pull/1871

To Reproduce
Steps to reproduce the behavior:

1. Configure request logging for a ktor server
2. Install 'XForwardedHeaderSupport`
3. print call.request.origin.remoteHost
4. Run the server behind a proxy
5. Send a request to the proxy
6. Observe that http is printed

Expected behavior
HTTP/1.1 should be printed

Hi,

we're sometimes (a few times a week) getting java.lang.NoClassDefFoundError from Netty. This often/always renders the server unresponsive. We've verified that the class that is not found is present in the (fat) jar that is deployed.

Ktor version: 1.3.2
Kotlin version: 1.3.72

Jul 08 08:59:42 dev java[24380]: Exception in thread "nioEventLoopGroup-4-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:498)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:96)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:1051)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
Jul 08 08:59:42 dev java[24380]:         at java.base/java.lang.Thread.run(Thread.java:834)
Jul 08 08:59:42 dev java[24380]: Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
Jul 08 08:59:42 dev java[24380]:         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
Jul 08 08:59:42 dev java[24380]:         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
Jul 08 08:59:42 dev java[24380]:         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
Jul 08 08:59:42 dev java[24380]:         ... 8 more

I'm not ruling out that we're doing something wrong, but there's no real giveaway into what that is exactly.

Thanks,
Ilja
JOIN, https://join.cc

I am getting a ResponseAlreadySentException when loading a static script via resource function on CIO server. It does not seem to affect the page loading, but pushes the following error to the log:

15:10:04.165 [DefaultDispatcher-worker-13] ERROR ktor.application - 200 OK: GET - /js/plotly.min.js
io.ktor.server.engine.BaseApplicationResponse$ResponseAlreadySentException: Response has already been sent
	at io.ktor.server.engine.BaseApplicationResponse.commitHeaders(BaseApplicationResponse.kt:44)
	at io.ktor.server.engine.BaseApplicationResponse.respondOutgoingContent$suspendImpl(BaseApplicationResponse.kt:127)
	at io.ktor.server.engine.BaseApplicationResponse.respondOutgoingContent(BaseApplicationResponse.kt)
	at io.ktor.server.cio.CIOApplicationResponse.respondOutgoingContent(CIOApplicationResponse.kt:111)
	at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invokeSuspend(BaseApplicationResponse.kt:307)
	at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invoke(BaseApplicationResponse.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(PipelineContext.kt:173)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:34)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.http.content.StaticContentKt$resource$1.invokeSuspend(StaticContent.kt:156)
	at io.ktor.http.content.StaticContentKt$resource$1.invoke(StaticContent.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:144)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:238)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:194)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:144)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(PipelineContext.kt:238)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:194)
	at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
	at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(PipelineContext.kt:144)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

The reproduced example is located here: https://github.com/mipt-npm/plotly.kt/blob/dev/examples/src/main/kotlin/dynamicServer.kt. The resource loading block is here.

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/601

Unhandled: POST - /sample00

sample code: https://github.com/BorzdeG/sample-ktor-kodein-controllers/blob/master/sample-00/src/main/kotlin/sample/sample00/Sam