Changelog 1.3 version

1.3.2

released 12th March 2020

Client

Allow to create threadless engine (CIO) for Ktor client

Add response text to the message of ResponseException and derived exceptions

HttpClient cancels parent coroutine despite try-catch block

How to use ContentEncoding?

Support Java HTTP Client

Support async DNS resolve in Client

Ktor Client HTTP Cache for invalid or blank Expires header

https://youtrack.jetbrains.com/issue/KTOR-364

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1892

Ktor Version and Engine Used (client or server and name) 1.3.2, client, io.ktor.client.features.cache.HttpCache

Describe the bug If a server returns a blank or an invalid Expires HTTP header, the ktor client crashes if the feature HttpCache is enabled install(HttpCache)

To Reproduce Steps to reproduce the behavior:

Write the following:

val articleUrl = Url("https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503")
val newsClient = HttpClient(Apache) {
    install(HttpCache)

    install(UserAgent) {
        agent = "Mozilla/5.0"
    }
}
try {
    val call: HttpStatement = newsClient.get(articleSimilarityRequest.articleUrl)
    val response: HttpResponse = call.receive()
    val requestUrl = response.request.url.toString()
    val rawHtml = response.readText()
    println(rawHtml)
} catch (cause: Throwable) {
    println("Unable to download article")
} finally {
    newsClient.close()
}

If you open the URL https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503 in the browser, you can see that the Expires response header is blank
An exception is thrown if you run the code:

2020-05-23 14:41:24.124 [DefaultDispatcher-worker-2 @request#48] ERROR a.s.api.util.service.DownloadService - Unable to download article
java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	(Coroutine boundary)
	at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:123)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:116)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
	at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:89)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
	at app.shwoo.api.util.service.DownloadService.downloadArticle$suspendImpl(DownloadService.kt:101)
	at app.shwoo.api.server.web.SimilarityResourceKt$similarity$1$4.invokeSuspend(SimilarityResource.kt:38)
	at io.ktor.routing.Routing.executeResult(Routing.kt:147)
	at io.ktor.routing.Routing.interceptor(Routing.kt:34)
	at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
	at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
	at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:292)
	at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
	at io.ktor.server.testing.TestApplicationEngine$handleRequest$pipelineJob$1.invokeSuspend(TestApplicationEngine.kt:290)
Caused by: java.lang.IllegalStateException: Failed to parse date: 
	at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
	at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
	at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
	at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Expected behavior It seems that the problem occurs in https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/features/cache/HttpCacheEntry.kt#L84. If the Expires header is blank an exception is thrown. The expected behavior is that the application handles the cache as expired if an invalid date is provided in the HTTP Expires header. Like described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires

Content-type header is missing in Request when using LogLevel.BODY

Can't compile project with Ktor JS client because of webpack errors

WebSocket throws a vague IOException when internet is disconnected

https://youtrack.jetbrains.com/issue/KTOR-623

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1119

Ktor Version

1.2.0-rc

Ktor Engine Used(client or server and name)

Client/CIO

JVM Version, Operating System and Relevant Context

OpenJDK 11.0.3, Arch Linux.

Feedback

When the internet is disconnected, a vague java.io.IOException is thrown twice, and it cannot be caught.

Exception in thread "pool-2-thread-3" java.io.IOException: Connection reset by peer
	at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
	at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
	at kotlinx.io.nio.ChannelsKt.read(Channels.kt:117)
	at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1.invokeSuspend(CIOReader.kt:75)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.ResumeModeKt.resumeMode(ResumeMode.kt:67)
	at kotlinx.coroutines.DispatchedKt.resume(Dispatched.kt:309)
	at kotlinx.coroutines.DispatchedKt.resumeUnconfined(Dispatched.kt:49)
	at kotlinx.coroutines.DispatchedKt.dispatch(Dispatched.kt:295)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:250)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:260)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:189)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:83)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:63)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:74)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

[Question] CPU 100% Apache Ktor Http Client

Disable system out for dropped connections

Uploading file without contenttype and filename cause server error

https://youtrack.jetbrains.com/issue/KTOR-772

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/458

Hi I use Apache HttpClient library to submit a multipart form to ktor service, Using the following Entity:

val entity = MultipartEntityBuilder.create()
                .addBinaryBody("image", img)

Then there will be exception at server.

kotlinx.io.core.MalformedUTF8InputException: Expected 2 more character bytes
	at kotlinx.io.core.ByteReadPacket.readUtf8(Packet.kt:488)
	at kotlinx.io.core.ByteReadPacket.readASCII(Packet.kt:267)
	at kotlinx.io.core.ByteReadPacket.readText(Packet.kt:224)
	at kotlinx.io.core.ByteReadPacket.readText$default(Packet.kt:220)
	at io.ktor.http.cio.CIOMultipartDataBase.partToData(CIOMultipartData.kt:115)
	at io.ktor.http.cio.CIOMultipartDataBase$partToData$1.doResume(CIOMultipartData.kt)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:41)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:41)
	at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:162)
	at kotlinx.coroutines.experimental.io.internal.MutableDelegateContinuation.run(MutableDelegateContinuation.kt:14)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

If I set contenttype and filename, the exception will dispear.

val entity = MultipartEntityBuilder.create()
                .addBinaryBody("image", img, ContentType.create("image/jpeg"), "ximg")

Add ability to override HttpAsyncResponseConsumer

Core

of multipart request, part.dispose should be called explicitly?

Is Ktor really that slow compared to other micro java frameworks?

Docs

"Using a Self-Signed Certificate" docs provide wrong dependency for 1.3.x

Documentation for kotlinx.serialization wrong and outdated

Prometheus Registry unavailable for Micrometer

Can I create a SOAP service with Ktor?

What is a "VOs"

Use hugo for building website?

Add previous and next navigation links to bottom of pages

https://youtrack.jetbrains.com/issue/KTOR-287

This issue was imported from GitHub issue: https://github.com/ktorio/ktorio.github.io/issues/105

Problem

As a new Ktor user I wanted to read the entire manual after getting started with the "Quick Start" section and before doing any serious coding.

It got frustrating reading the "Servers" section because once I'd reached the end of a long page, I'd have to scroll back towards the top, find the active navigation link, and then click on the next one.

Solution

Add previous and next links at the bottom of each page that has a category. I have a local branch that implements this approach that I could merge and use to submit a PR after I finish it.

Update

I have a working branch here: https://github.com/kensiprell/ktorio.github.io/tree/prev-next. I don't want to create a PR until someone takes a look because it touches so much.

_layouts/page.html:

        {% if page.status == 'todo' %}
          Coming soon...
        {% else %}
          {{ content }}
        {% endif %}

        {% if page.category != null %}
            {% include prev-next.html %}
        {% endif %}
      </div>
    </div>
  </div>
</div>

_includes/prev-next.html:

{% assign entries = site.pages | sort: "path" %}
{% for entry in entries %}
    {% if entry.url == page.url %}
        {% assign prevIndex = forloop.index0 | minus: 1 %}
        {% assign nextIndex = forloop.index0 | plus: 1 %}
        <div class="container-fluid col-sm-12" style="padding-top:2rem;border-top:1px solid #ddd;">
            <div class="row">
                <div class="col-sm-6">
                    {% if prevIndex %}
                        {% if entries[prevIndex].category == page.category %}
                            <a href="{{ entries[prevIndex].url }}">
                                &larr; {{ entries[prevIndex].title }}
                            </a>
                        {% else %}
                            First page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
                <div class="col-sm-6 text-right">
                    {% if nextIndex %}
                        {% if entries[nextIndex].category == page.category %}
                            <a href="{{ entries[nextIndex].url }}">
                                {{ entries[nextIndex].title }} &rarr;
                            </a>
                        {% else %}
                            Last page in {{ page.category | capitalize }}
                        {% endif %}
                    {% endif %}
                </div>
            </div>
        </div>
    {% endif %}
{% endfor %}

Here are some images showing the proof of concept.

First page in the "Kotlinx" category:

prev-next-1

Second page in the "Kotlinx" category:

prev-next-2

Last page in the "Kotlinx" category:

prev-next-3

The real work involves making children directory names match the file names and then adding permalinks (see image below). I used Jekyll for the first time two days ago, so I'm not 100% certain that my approach won't have negative ramifications elsewhere. However, I haven't noticed any issues running it locally on my computer.

prev-next

metrics prometheus better documentation

Custom JSON mapping with Jackson

Add support for the SameSite cookie attribute

Process

Ktor 1.3.2 missing from Maven Central

Samples

Corrupted files with low connection rate (Jetty)

Routing: get matcher has higher priority than param matcher on the same level

ApplicationCall.push fails under HTTP/1.1

Server

Support routing prefix

Exception after WebSocketSession.close() invocation.

java.lang.NoClassDefFoundError using Netty backend

https://youtrack.jetbrains.com/issue/KTOR-19

Hi,

we're sometimes (a few times a week) getting java.lang.NoClassDefFoundError from Netty. This often/always renders the server unresponsive. We've verified that the class that is not found is present in the (fat) jar that is deployed.

Ktor version: 1.3.2 Kotlin version: 1.3.72

Jul 08 08:59:42 dev java[24380]: Exception in thread "nioEventLoopGroup-4-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:498)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:96)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:1051)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
Jul 08 08:59:42 dev java[24380]:         at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
Jul 08 08:59:42 dev java[24380]:         at java.base/java.lang.Thread.run(Thread.java:834)
Jul 08 08:59:42 dev java[24380]: Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
Jul 08 08:59:42 dev java[24380]:         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
Jul 08 08:59:42 dev java[24380]:         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
Jul 08 08:59:42 dev java[24380]:         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
Jul 08 08:59:42 dev java[24380]:         ... 8 more

I'm not ruling out that we're doing something wrong, but there's no real giveaway into what that is exactly.

Thanks, Ilja JOIN, https://join.cc

Parent coroutine context cancellation sometimes leads to hanging coroutine

https://youtrack.jetbrains.com/issue/KTOR-339

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1947

Ktor Version and Engine Used (client or server and name) ktor-server-jetty and ktor-websockets 1.3.2 coroutines version 1.3.4

Describe the bug Sometimes my tests hang on post-test test job cancellation. That job contains both client and server coroutine scopes/jobs as decdendants.

Coroutines dump (printed after test timeout) gives this:

[15:17:55] :			 [gant] [junit] Coroutine "jetty-call-handler#6995":StandaloneCoroutine{Cancelling}@258a8584, state: SUSPENDED
[15:17:55] :			 [gant] [junit] 	at io.ktor.server.jetty.JettyKtorHandler$handle$2.invokeSuspend(JettyKtorHandler.kt:125)
[15:17:55] :			 [gant] [junit] 	at    (Coroutine creation stacktrace. ( )
[15:17:55] :			 [gant] [junit] 	at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)
[15:17:55] :			 [gant] [junit] 	at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
[15:17:55] :			 [gant] [junit] 	at kotlinx.coroutines.BuildersKt.launch$default(Unknown Source)
[15:17:55] :			 [gant] [junit] 	at io.ktor.server.jetty.JettyKtorHandler.handle(JettyKtorHandler.kt:85)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.Server.handle(Server.java:500)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
[15:17:55] :			 [gant] 
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
[15:17:55] :			 [gant] [junit] 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)

To Reproduce Unfortunately, the bug can't be reproduced locally even with the existing tests, happens only on CI. No minimal sample yet.

Expected behavior No suspended coroutines after a parent-induced cancellation.

Class cast exception on web-socket cancelation

https://youtrack.jetbrains.com/issue/KTOR-349

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1929

Ktor Version and Engine Used (client or server and name) ktor-cio server 1.3.2

Describe the bug I am getting an internal error when force-closing server with opened web-socket connection Here is the stack trace:

13:32:20.839 [DefaultDispatcher-worker-4] ERROR io.ktor.server.cio.HttpServer - Unhandled exception caught for CoroutineName(http-pipeline-writer)
kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[ExperimentalCoroutineDispatcher@45e6d6a4[scheduler = DefaultDispatcher@6341301e[Pool Size {core = 10, max = 1024}, Worker States {CPU = 6, blocking = 0, parked = 8, dormant = 0, terminated = 0}, running workers queues = [0c, 0c, 0c, 0c, 0c, 0c], global CPU queue size = 5, global blocking queue size = 0, Control State {created workers= 14, blocking tasks = 0, CPUs acquired = 6}]], Continuation at io.ktor.server.cio.backend.ServerPipelineKt$startServerConnectionPipeline$1$outputsActor$1.invokeSuspend(ServerPipeline.kt:49)@2482cf33]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
	at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
	at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
	at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
	... 4 common frames omitted

CORS - request with non-simple content type not failing

ktor not closing user connections

https://youtrack.jetbrains.com/issue/KTOR-368

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1885

Ktor Version and Engine Used (client or server and name) 1.3.2, Netty server, routes

ktor_version=1.3.2, kotlin_version=1.3.70

implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version")
    implementation("io.ktor:ktor-server-netty:$ktor_version")
    implementation("ch.qos.logback:logback-classic:$logback_version")
    testImplementation("io.ktor:ktor-server-tests:$ktor_version")

Describe the bug I migrated one of my project form php to ktor and deployed it in production for about 5k users. After one day server stopped working and linux file opened limits exceed which was 20k and when i checked with command ss -ant | grep :8383 | wc -l connections were 20k+ but i don't have users more than 5k which were not all active. Then i switched my floating ip to old php server and no more request to this server next day when i checked again with command ss -ant | grep :8383 | wc -l connections were still 20k+ even no more new request were coming and all connections were in ESTAB state.

So after that i created new project which was empty with ktor, kotlin dsl i mentioned its dependencies above and run that on that server Then i run this command watch -n 1 'ss -ant | grep {MY_IP}' on server to filter connections for my ip first it was no connection from my ip on port 8383 then i send single request from postman on SERVER_IP:8388 which was simple get request and response was from call.respond("Testing") and on server terminal it show me connection on 8383 with my ip. In postman request was complete and i got result. Then i closed postman and keep checking when it will close my connection from server. I checked that after an hour still connection was not closed and when i requested again it was still using that same connection. So it was reusing connection and not closing even after some hours. I'm not sure but may be error with keep alive or http2. I'm not sending any header i request.

I didn't tried any other ktor or kotlin version and I'm not able to reproduce it local. I'm running jar file generated with shadowjar on linux server with systemd service.

To Reproduce

create new project with ktor version 1.3.2 and kotlin 1.3.70 with GradleKotlinDsl

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(Routing) {
        get("/") {
            call.respond("Testing")
        }
    }
}

create jar file with shadowjar

tasks {
    named<ShadowJar>("shadowJar") {
        archiveBaseName.set("shadow")
        mergeServiceFiles()
        manifest {
            attributes(mapOf("Main-Class" to mainClasses))
        }
        transform(com.github.jengelman.gradle.plugins.shadow.transformers.ServiceFileTransformer::class.java) {
            setPath("META-INF/services")
            include("org.eclipse.jetty.http.HttpFieldPreEncoder")
        }
        archiveFileName.set("example.jar")
    }
}

upload on ubuntu server and create its systemd service and run with systemctl start example service file:

[Unit]
Description=example
After=syslog.target

[Service]
User=root
LimitNOFILE=20536
ExecStart=/var/www/example/example-script
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

example-script:

#!/bin/sh
cd /var/www/example
java -jar exmaple.jar

Expected behavior When user make request to http://SERVER_IP:APP_PORT it should close it after request complete but it keep opened it until we will stop server. Checking with ss -ant | grep {CLIENT_PUBLIC_IP}

Screenshots This is screenshot which i get after an hour when made last request in screenshot last connection with port 8383 that is from my machine and it is still not closed. Screenshot 2020-05-20 at 9 32 50 PM

Scheme instead of version in OriginConnectionPoint

Get client certificate information from request

Occasionally Ktor utilizes 100% CPU without any load after failed request.

https://youtrack.jetbrains.com/issue/KTOR-531

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1433

Ktor Version and Engine Used (client or server and name) Our application uses both - Ktor server and client. Versions are ktor: 1.2.3 kotlin: 1.3.50 server's engine: Netty client's engine: CIO

        // http server
        implementation "io.ktor:ktor-server-core:$ktor_version"
        implementation "io.ktor:ktor-server-netty:$ktor_version"
        implementation "io.ktor:ktor-server-host-common:$ktor_version"
        implementation "io.ktor:ktor-server-sessions:$ktor_version"
        implementation "io.ktor:ktor-auth:$ktor_version"
        implementation "io.ktor:ktor-jackson:$ktor_version"
        implementation "io.ktor:ktor-auth-jwt:$ktor_version"
        implementation "io.ktor:ktor-locations:$ktor_version"
        // http client
        implementation "io.ktor:ktor-client-core:$ktor_version"
        implementation "io.ktor:ktor-client-core-jvm:$ktor_version"
        implementation "io.ktor:ktor-client-apache:$ktor_version"
        implementation "io.ktor:ktor-client-cio:$ktor_version"
        implementation "io.ktor:ktor-client-json:$ktor_version"
        implementation "io.ktor:ktor-client-json-jvm:$ktor_version"
        implementation "io.ktor:ktor-client-jackson:$ktor_version"
        implementation "io.ktor:ktor-client-logging:$ktor_version"
        implementation "io.ktor:ktor-client-logging-jvm:$ktor_version"
        implementation "io.ktor:ktor-client-auth:$ktor_version"
        implementation "io.ktor:ktor-client-auth-jvm:$ktor_version"

Run on a VPC: Linux Debian 9.11

Describe the bug Monitoring reported that our server consumes 100%. The server does nothing (it's test environment and issue happened later evening) but shows high CPU usage. In jstack, I found only one suspicious thread:

"Thread-8@16328" daemon prio=10 tid=0x6f nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at sun.nio.ch.EPoll.wait(EPoll.java:-1)
      at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
      at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
      - locked <0x3ff5> (a sun.nio.ch.EPollSelectorImpl)
      - locked <0x403d> (a sun.nio.ch.Util$2)
      at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:146)
      at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:81)
      at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt:-1)
      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
      at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.lang.Thread.run(Thread.java:834)

Strace shows that some thread is performing a lot of epolls (all with same fd) and these polls are finished at once:

[pid  5938] 23:07:27.637413 epoll_wait(63, [], 1024, 0) = 0 <0.000014>
[pid  5938] 23:07:27.637473 epoll_wait(63, [], 1024, 0) = 0 <0.000016>
[pid  5938] 23:07:27.637538 epoll_wait(63, [], 1024, 0) = 0 <0.000015>
[pid  5938] 23:07:27.637598 epoll_wait(63, [], 1024, 0) = 0 <0.000043>
[pid  5938] 23:07:27.637687 epoll_wait(63, [], 1024, 0) = 0 <0.000016>
[pid  5938] 23:07:27.637739 epoll_wait(63, [], 1024, 0) = 0 <0.000015>
[pid  5938] 23:07:27.637799 epoll_wait(63, [], 1024, 0) = 0 <0.000015>
[pid  5938] 23:07:27.637860 epoll_wait(63, [], 1024, 0) = 0 <0.000016>

lsof shows: java 3405 root 63u a_inode 0,13 0 17139 [eventpoll]

To Reproduce According to monitoring system, usage spike happened at same time then an user tried to logout using expired session. In our system it will lead to throwing an exception and handing it with status page:

    install(StatusPages) {
    ...
        Exceptions.apply {
            httpExceptions(testing)
        }
    }

    @KtorExperimentalAPI
    fun StatusPages.Configuration.httpExceptions(testing: Boolean) {
        exception<HttpError> {
            logger.warn("${this.context.request.uri} - failed due to ${it.message}${it.cause?.let {"(caused by $it)"}}", it)
            if (it.code == HttpStatusCode.Unauthorized) {
                call.unauthorized(it.body)
            } else {
                it.body?.let { body ->
                    call.respond(it.code, body)
                } ?: call.respond(it.code)
            }
        }
        ... 

@KtorExperimentalAPI
suspend fun ApplicationCall.unauthorized(maybeError: HttpErrorBody? = null): Unit {
    // set WWW-Authenticate (as per RFC required for 401 status)
    val realm = application.environment.config.property("authentication.realm").getString()
    val header = HttpAuthHeader.Parameterized(AuthenticationScheme, mapOf(HttpAuthHeader.Parameters.Realm to realm))
    response.headers.append(HttpHeaders.WWWAuthenticate, header.toString())
    // clear session
    sessions.clear<MySessionCookie>()
    // return status code and body
    maybeError?.let { body ->
        respond(HttpStatusCode.Unauthorized, body)
    } ?: respond(HttpStatusCode.Unauthorized)
}

In log, we have:

2019-11-08 19:31:58,277 DEBUG [nioEventLoopGroup-4-1][172.26.0.8][REQ-496] auth - session 8670142f-97a2-4035-9f17-e62774e0a7c5 has expired
2019-11-08 19:31:58,277 INFO  [nioEventLoopGroup-4-1][172.26.0.8][REQ-496] application - finished POST /web/v1/logout with null
2019-11-08 19:31:58,278 WARN  [nioEventLoopGroup-4-1][172.26.0.8][REQ-496] application - /web/v1/logout - failed due to SessionExpired com.example.http.HttpError$UnauthorizedAccess: SessionExpired
        at com.example.ApplicationKt$module$12$$special$$inlined$session$lambda$1.invokeSuspend(Application.kt:297)
        at com.example.ApplicationKt$module$12$$special$$inlined$session$lambda$1.invoke(Application.kt)
        at com.example.ApplicationKt$module$12$$special$$inlined$session$1.invokeSuspend(SessionAuth.kt:156)
        at com.example.ApplicationKt$module$12$$special$$inlined$session$1.invoke(SessionAuth.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
        at io.ktor.auth.Authentication.processAuthentication(Authentication.kt:228)
        at io.ktor.auth.Authentication$interceptPipeline$2.invokeSuspend(Authentication.kt:123)
        at io.ktor.auth.Authentication$interceptPipeline$2.invoke(Authentication.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
        at io.ktor.routing.Routing.executeResult(Routing.kt:147)
        at io.ktor.routing.Routing.interceptor(Routing.kt:34)
        at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
        at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:106)
        at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:98)
        at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
        at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:180)
        at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:97)
        at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:137)
        at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
         at io.ktor.features.CallLogging$Feature$install$1$invokeSuspend$$inlined$withMDC$1.invokeSuspend(CallLogging.kt:226)
        at io.ktor.features.CallLogging$Feature$install$1$invokeSuspend$$inlined$withMDC$1.invoke(CallLogging.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
        at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:156)
        at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
        at io.ktor.features.CallLogging$Feature$install$1.invokeSuspend(CallLogging.kt:230)
        at io.ktor.features.CallLogging$Feature$install$1.invoke(CallLogging.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
        at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
        at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:36)
        at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
        at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:111)
        at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
        at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
        at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
        at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:26)
        at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:20)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:515)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)

To me, It looks very similar to: https://github.com/ktorio/ktor/issues/1041 But it was about client, while we've issue with a server.

Expected behavior Doesn't consume CPU without load.

Multiple matching paths do not choose higher quality path (component with more static values) when the lower quality one is authenticated

"HTTP Exception 404" for POST from Location

ktor-client doesn't send body sometimes

[Regression Bug 0.9.4] Adding call.respond to pipeline throws java.lang.reflect.InvocationTargetException

Websockets and Netty weird exception

Setting multiple cache control directives is impossible with current API

Sensible defaults for compression feature

embeddedServer for CIO and Netty inconsistency

MultiPartData.readAllParts() throws java.io.IOException when multipart list is empty

https://youtrack.jetbrains.com/issue/KTOR-767

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/482

Problem description: When receiving an empty multipart-form request on a POST route, the request handler throws an IOException, preventing graceful handling or the display of an error message. Instead, a 500 error message is returned.

Minimum working example:

import io.ktor.application.*
import io.ktor.content.readAllParts
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.post
import io.ktor.routing.routing

fun main(args: Array<String>): Unit = io.ktor.server.netty.DevelopmentEngine.main(args)

fun Application.module() {
    routing {
        post("/add") {
            if(call.request.isMultipart()) {
                val multipart = call.receiveMultipart()
                val formItems = multipart.readAllParts() // <-- this breaks when sent an empty request
                call.respondText("recvd: $multipart")
            }
            else {
                call.respond("no multipart, fix request!")
            }
        }
    }
}

Send a request with an empty multipart body to see the issue:

curl -X POST \
  http://localhost:8080/add \
  -H 'Cache-Control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'

Error Message:

22:57:11.866 [nettyCallPool-4-4] ERROR Application - Unhandled: POST - /add
java.io.IOException: Broken delimiter occurred
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.doResume(Delimited.kt:57)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
	at kotlinx.coroutines.experimental.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt:1746)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiterSuspend(Delimited.kt:55)
	at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiter(Delimited.kt:50)
	at io.ktor.http.cio.MultipartKt.boundary(Multipart.kt:89)
	at io.ktor.http.cio.MultipartKt$parseMultipart$1.doResume(Multipart.kt:158)
	at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
	at kotlinx.coroutines.experimental.DispatchedKt.resumeCancellable(Dispatched.kt:209)
	at kotlinx.coroutines.experimental.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:35)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:111)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce(Produce.kt:95)
	at kotlinx.coroutines.experimental.channels.ProduceKt.produce$default(Produce.kt:88)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:145)
	at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:138)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:33)
	at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:31)
	at io.ktor.server.engine.DefaultTransformKt.multiPartData(DefaultTransform.kt:70)
	at io.ktor.server.engine.DefaultTransformKt.access$multiPartData(DefaultTransform.kt:1)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.doResume(DefaultTransform.kt:33)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:64)
	at io.sebi.ApplicationKt$module$1$1.doResume(application.kt:31)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.routing.Routing.executeResult(Routing.kt:100)
	at io.ktor.routing.Routing.interceptor(Routing.kt:25)
	at io.ktor.routing.Routing$Feature$install$1.doResume(Routing.kt:66)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
	at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt:51)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.doResume(DefaultEnginePipeline.kt:66)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
	at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
	at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.doResume(NettyApplicationCallHandler.kt:31)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt:10)
	at kotlinx.coroutines.experimental.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:44)
	at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:113)
	at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch(Builders.common.kt:72)
	at kotlinx.coroutines.experimental.BuildersKt.launch(Unknown Source)
	at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:64)
	at kotlinx.coroutines.experimental.BuildersKt.launch$default(Unknown Source)
	at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
	at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:844)

Confusing log message about failed session lookup

raw socket bug?

Brotli Support

HTTP/2 push fails with netty engine

https://youtrack.jetbrains.com/issue/KTOR-800

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/350

The problem reproduces with http2-push from https://github.com/ktorio/ktor-samples

Run ./gradlew :http2-push:run
Navigate to https://localhost:8443/

Resulting exception:

2018-03-07 14:47:07.089 [nettyWorkerPool-3-2] WARN  i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
2018-03-07 14:47:07.093 [nettyWorkerPool-3-2] WARN  i.n.util.concurrent.DefaultPromise - An exception was thrown by io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete()
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
        at io.ktor.server.netty.http2.NettyHttp2Handler$startHttp2PushPromise$1.operationComplete(NettyHttp2Handler.kt:109)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
        at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163)
        at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:108)
        at io.ktor.server.netty.http2.NettyHttp2ApplicationResponse$push$1.run(NettyHttp2ApplicationResponse.kt:49)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Stream no longer exists: 3
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.requireStream(DefaultHttp2ConnectionEncoder.java:343)
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writePushPromise(DefaultHttp2ConnectionEncoder.java:279)
        at io.ktor.server.netty.http2.NettyHttp2Handler.startHttp2PushPromise$ktor_server_netty(NettyHttp2Handler.kt:104)
        ... 7 common frames omitted

Receive JSON formatted POST data into ValuesMap with Gson

Sample using Swagger

Support Sealed Classes inside Session-Objects

Test Infrastructure

TestApplicationRequest returns status code "400 Bad Request" when server handling ContentNegotiation Jackson on a post request (application/json) but works well from a curl request

https://youtrack.jetbrains.com/issue/KTOR-18

[General Information]

ktor_version=1.3.2
kotlin_version=1.3.70
server engine : io.ktor.server.netty.EngineMain

Short version of ApplicationTest.kt

class ApplicationTest {
    @Test
    fun postPlace() {
        withTestApplication({ module(testing = true) }) {
            handleRequest(HttpMethod.Post, "/places") {
                addHeader(HttpHeaders.ContentType, ContentType.Application.Json.contentType)
                setBody(json { "title" to "Yammiii" }.toString())
            }.apply {
                assertEquals(HttpStatusCode.BadRequest, response.status(), "Missing mandatory fields")
            }
            handleRequest(HttpMethod.Post, "/places") {
                addHeader(HttpHeaders.ContentType, ContentType.Application.Json.contentType)
                setBody(json {
                    "title" to "Yammiii"
                    "country" to "Belgique"
                    "city" to "Bruxelles"
                }.toString())
            }.apply {
                assertEquals(HttpStatusCode.Created, response.status(), "all mandatory fields are provided")
            }
        }
    }
}

The lineassertEquals(HttpStatusCode.Created, response.status(), "all mandatory fields are provided") fails with a 400 Bad Request while the same curl (on Windows) request works : curl -d "{\"title\":\"value1\", \"country\":\"value2\", \"city\":\"value3\"}" -H "Content-Type:application/json" -X POST http://localhost:8080/places

Short version of 'Application.kt'

@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {

    install(DefaultHeaders)
    install(CallLogging)
    install(WebSockets)

    install(ContentNegotiation) {
        jackson {

        }
    }

    DatabaseFactory.init()

    val beehivePlaceService = BeehivePlaceService()
    ...

    install(Routing) {
        beehivePlace(beehivePlaceService)
        ...
    }
}

fun Route.beehivePlace(beehivePlaceService: BeehivePlaceService) {

    route("/places") {

         ...

        post("/") {
            val beehivePlace = call.receive<NewBeehivePlace>()
            call.respond(HttpStatusCode.Created, beehivePlaceService.addBeehivePlace(beehivePlace))
        }

        ....

    }


}

BeehivePlace.kt

package com.sokarcreative.beefamily.model

import org.jetbrains.exposed.sql.Table

...

data class NewBeehivePlace(
    val id: Int? = null,
    val title: String = "",
    val country: String? = null,
    val city: String? = null,
    val postalCode: Int? = null,
    val streetAddress: String? = null,
    val streetNumber: String? = null,
    val streetBox: String? = null,
    val phoneNumber: String? = null,
    val websiteUrl: String? = null,
    val facebookUrl : String? = null,
    val twitterUrl: String? = null,
    val instagramUrl: String? = null,
    val mailAddress: String? = null
)

The 400 Bad Request is thrown when call.receive is called at this line : val beehivePlace = call.receive<NewBeehivePlace>()

Why is this post request working on a simple curl request and not in the TestApplicationRequest ?

I started calls directly from the test application class and I didn't understand why it isn't working, I thought it was a problem of mine before testing in a curl request and see that it was actually working on it.

Any help would be really appreciated :)

A beginner on ktor,

Other

Unsubscribe from events

Support routing data in content negotiation

Send 100 Continue response only when getting a request to receive `IncomingContent`

Thymeleaf ExpressionObjecs invalied , ex #httpServletRequest

Header Content-Length is controlled by the engine and cannot be set explicitly

https://youtrack.jetbrains.com/issue/KTOR-310

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1993

When using the Ktor http client (ktor 1.3.2) for PUT request without payload the Content-Length is not set, but it should be.

But, setting the Content-Length explicitly in the code with "header(HttpHeaders.ContentLength, 0)" cause the following exception:

io.ktor.http.UnsafeHeaderException: Header Content-Length is controlled by the engine and cannot be set explicitly
	at io.ktor.client.engine.HttpClientEngineKt.validateHeaders(HttpClientEngine.kt:140)
	at io.ktor.client.engine.HttpClientEngineKt.access$validateHeaders(HttpClientEngine.kt:1)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:62)
	at io.ktor.client.engine.HttpClientEngine$install$1.invoke(HttpClientEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:115)
	at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
	at io.ktor.client.features.HttpSend$Feature$install$1.invoke(HttpSend.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
	at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invoke(HttpRequestLifecycle.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:318)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:163)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:183)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
	at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:58)

Client for OkHttp 3.12.x

GSON Support turns List<Long> into List<Double> during deserialization

standalone ktor server (eg.: binary linux, or exe windows)

CIO, Canceling static file request; Connection reset exception

https://youtrack.jetbrains.com/issue/KTOR-323

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1975

Ktor Version and Engine Used (client or server and name) 1.3.2, CIO server

Describe the bug CIO engine throws an exception when a static file HTTP request is canceled.

Exception in thread "DefaultDispatcher-worker-3" java.net.SocketException: Connection reset
	at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:345)
	at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:376)
	at io.ktor.utils.io.nio.ChannelsKt.read(Channels.kt:135)
	at io.ktor.network.sockets.CIOReaderKt$attachForReadingDirectImpl$1$1$1.invokeSuspend(CIOReader.kt:90)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:175)
	at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:137)
	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:306)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:316)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:248)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKey(SelectorManagerSupport.kt:84)
	at io.ktor.network.selector.SelectorManagerSupport.handleSelectedKeys(SelectorManagerSupport.kt:64)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:73)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

To Reproduce Steps to reproduce the behavior:

Write the following

fun main() = runBlocking<Unit> {
    embeddedServer(CIO, port = 8080, module = Application::module).start(wait = true)
}

fun Application.module() {
    routing {
        static {
            resource("/hello.txt") // Some big file so you have time to cancel the request
        }
    }
}

Run using browser/curl/etc and access localhost:8080/hello.txt; Cancel the request before ending.
See the error logged.

Other things to mention

This line is logged after the error. So, the server probably sees the request cancellation.

18:28:42.835 [DefaultDispatcher-worker-1] INFO ktor.application - 200 OK: GET - /hello.txt, cancelled

This won't happen with Netty server engine (it does the expected behavior).

Expected behavior I guess, a simple INFO log about request cancellation and nothing more.

CIO client silently failing on connection closed

Client: URL encode / escaping is wrong

Can't use ktor-ktor-client-core with webpack

Failed to build ktor locally: java.lang.OutOfMemoryError: GC overhead limit exceeded

AndroidClientEngine set a Content-Lenght: 2 on a GET request with an EmptyContent as body

TestHttpClientEngine KNPE

Ktor Client Null Pointer Exception (HTTPS)

Ktor should resolve trailing slash as a different URL

Ktor 1.3.1 Fails File Upload with MalformedInputException

https://youtrack.jetbrains.com/issue/KTOR-391

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1845

Hi. I am using Ktor Client with version 1.3.1 for MPP

I have been using Ktor version 1.2.6 in my MPP project with Kotlin version 1.3.61 and it was working fine, so I was able to upload files without a problem. Later I update Kotlin version to 1.3.72 which forced me to update Ktor version 1.3.1 as version 1.2.6 was compiled with Kotlin version 1.3.61 which IDE can not compile with Kotlin plugin 1.3.72. After that file upload call started to give me the following exception.

2020-05-02 17:35:35.186 14564-14671/com.inivo.client.debug E/AndroidRuntime: FATAL EXCEPTION: ktor-android-dispatcher-worker-2
    Process: com.inivo.client.debug, PID: 14564
    io.ktor.utils.io.charsets.MalformedInputException: Input length = 1
        at io.ktor.utils.io.charsets.CharsetJVMKt.throwExceptionWrapped(CharsetJVM.kt:323)
        at io.ktor.utils.io.charsets.CharsetJVMKt.decode(CharsetJVM.kt:199)
        at io.ktor.utils.io.charsets.EncodingKt.decode(Encoding.kt:101)
        at io.ktor.utils.io.core.StringsKt.readText(Strings.kt:251)
        at io.ktor.utils.io.core.StringsKt.readText$default(Strings.kt:250)
        at io.ktor.client.features.logging.Logging$logRequestBody$2$1.invokeSuspend(Logging.kt:111)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:69)
        at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:184)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:307)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:317)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:249)
        at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2210)
        at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:930)
        at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:544)
        at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:606)
        at io.ktor.utils.io.ByteBufferChannel.readAvailable(Unknown Source:0)
        at io.ktor.utils.io.jvm.javaio.WritingKt.copyTo(Writing.kt:21)
        at io.ktor.utils.io.jvm.javaio.WritingKt.copyTo$default(Writing.kt:12)
        at io.ktor.client.engine.android.AndroidClientEngineKt.writeTo(AndroidClientEngine.kt:112)
        at io.ktor.client.engine.android.AndroidClientEngine.execute(AndroidClientEngine.kt:79)
        at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:83)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

I am uploading a file like this.

override suspend fun sendImageAsync(
        token: String,
        file: File,
        fileType: String,
        id: Long
    ): Unit = client.request {
        fullUrl("${RestApi.SEND_IMAGE}/${id}")
        method = HttpMethod.Post
        body = MultiPartFormDataContent(
            formData {
                append(
                    "image",
                    file.fileData.bytes,
                    headers = Headers.build {
                        append(HttpHeaders.ContentType, "image/jpg")
                        append(
                            HttpHeaders.ContentDisposition,
                            "filename=${file.name}"
                        )
                        append("Connection", "Close")
                    }
                )
            }
        )
        header(HttpHeaders.Connection, "close")
        header(HttpHeaders.Authorization, makeAuthToken(token))
    }

Here file data is just an object which keeps File data as a byte array so by that I can convert image to byte array (either from Android / IOS) and upload it.

Thank you very much for your help in advanced.

URLBuilder to append paths to currently encoded path

Windows Curl invalid mutability exception

https://youtrack.jetbrains.com/issue/KTOR-406

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1820

Ktor Version and Engine Used (client or server and name) OS: Windows 10 (64 bit) Client version: CURL (msys64) ktor version: 1.3.2 Kotlin version: 1.3.72

Describe the bug Sending a request that is not successful (status code != 200) throws InvalidMutabilityException

To Reproduce

    val client = HttpClient(Curl)
    client.get<String>("https://any-url-that-returns-status-code-not-200.com")

Expected behavior Code throws ClientRequestException, like it does on JVM

Screenshots

Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlin.collections.HashMap@1240b58
        at kfun:kotlin.Throwable.<init>(kotlin.String?;kotlin.Throwable?)kotlin.Throwable (000000000045e120)
        at kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable (000000000045e410)
        at kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception (0000000000457800)
        at kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException (0000000000457340)
        at kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException (00000000004848e0)
        at ThrowInvalidMutabilityException (0000000000485d50)
        at MutationCheck (00000000008b3800)
        at kfun:kotlin.collections.HashMap.<set-length>#internal (0000000000468fc0)
        at kfun:kotlin.collections.HashMap.addKey$stdlib(K)kotlin.Int (000000000046cea0)
        at kfun:kotlin.collections.HashMap.put(K;V)V? (000000000046a000)
        at kfun:io.ktor.util.AttributesNative.put#internal (0000000000660e50)
        at kfun:io.ktor.client.features.$addDefaultResponseValidation$lambda-1$lambda-0COROUTINE$174.invokeSuspend#internal (00000000006b6db0)
        at kfun:io.ktor.client.features.$addDefaultResponseValidation$lambda-1$lambda-0COROUTINE$174.invoke#internal (00000000006b7980)
        at kfun:io.ktor.client.features.HttpCallValidator.$validateResponseCOROUTINE$180.invokeSuspend#internal (00000000006bfb00)
        at kfun:io.ktor.client.features.HttpCallValidator.validateResponse#internal (00000000006c0210)
        at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$179.invokeSuspend#internal (00000000006c2800)
        at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$179.invoke#internal (00000000006c3430)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (000000000065d4e0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (000000000065cf50)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal (000000000065d2a0)
        at kfun:io.ktor.util.pipeline.Pipeline.execute(TContext;TSubject)TSubject (00000000006583d0)
        at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$169.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (00000000006aa240)
        at kfun:io.ktor.client.call.HttpClientCall.receive(io.ktor.client.call.TypeInfo)kotlin.Any (00000000006aaf50)
        at kfun:sample.$main$lambda-0COROUTINE$0.invokeSuspend#internal (000000000084a030)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0000000000478be0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (000000000065dc30)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (000000000065d4e0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (000000000065f700)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0000000000478be0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (000000000065dc30)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (000000000065d4e0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (000000000065f700)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0000000000478be0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (000000000065dc30)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (000000000065d4e0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (000000000065f700)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0000000000478be0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (000000000065dc30)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (000000000065d4e0)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (000000000065f700)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0000000000478be0)
        at kfun:kotlinx.coroutines.DispatchedTask.run() (000000000057ee30)
        at kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()kotlin.Long (000000000053d3f0)
        at kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal (000000000058cd40)
        at kfun:kotlinx.coroutines.runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>){0<kotlin.Any?>}Generic (000000000058c130)
        at kfun:kotlinx.coroutines.runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,T>;kotlin.Int){0<kotlin.Any?>}Generic (000000000058c9f0)
        at kfun:sample.main() (0000000000849e60)
        at Konan_start (000000000084cc10)
        at Init_and_run_start (00000000008920c0)
        at __tmainCRTStartup (0000000000401180)
        at mainCRTStartup (00000000004014f0)
        at  (00007fffccf67bc0)
        at  (00007fffcd22ce30)

MalformedInputException: Input length = 1, when uploading an image multipart

https://youtrack.jetbrains.com/issue/KTOR-425

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1766

Ktor Version and Engine Used (client or server and name) Ktor 1.3.2, Kotlin 1.3.70, Netty server, ContentNegotiation, Jackson parser, Exposed.

I simply used the Ktor project generator for Gradle, and checked a few boxes for modules as mentioned above. After which I managed to connect a database & a few endpoints, but my core endpoint for uploading images doesn't work.

Describe the bug When trying to upload a simple image using Retrofit & Multipart support in Android, I get a MalformedInputException on my server.

Retrofit/Android code:

@Multipart
@POST("/api/upload")
suspend fun uploadImages(@Part("image") imageFile: RequestBody): UploadResponse

Upload code:

val requestBody: RequestBody = MultipartBody.Builder()
        .setType(MultipartBody.FORM)
        .addFormDataPart("title", "Image Upload")
        .addFormDataPart(
            "image", // part name
            file.path.split("/").last(), // file name
            file.asRequestBody("image/jpg".toMediaType()) // Content-Type: image/jpg
        )
        .build()

    return apiService.uploadImages(requestBody)

Using the new way of creating RequestBody objects in OkHttp. I followed the documentation to receive the multipart data, and I've formed a MalformedInputException.

To Reproduce Steps to reproduce the behavior:

Use Retrofit to Upload an image to a Ktor server.
call receiveMultipart() to receive MP on the Ktor side.
See error

Expected behavior Multipart data should be received correctly.

Stack trace:

[nioEventLoopGroup-4-1] ERROR Application - Unhandled: POST - /api/upload
io.ktor.utils.io.charsets.MalformedInputException: Input length = 1
	at io.ktor.utils.io.charsets.CharsetJVMKt.throwExceptionWrapped(CharsetJVM.kt:323)
	at io.ktor.utils.io.charsets.CharsetJVMKt.decodeImplSlow(CharsetJVM.kt:289)
	at io.ktor.utils.io.charsets.CharsetJVMKt.decodeExactBytes(CharsetJVM.kt:254)
	at io.ktor.utils.io.core.StringsKt.readTextExactBytes(Strings.kt:293)
	at io.ktor.utils.io.core.StringsKt.readTextExactBytes$default(Strings.kt:292)
	at io.ktor.utils.io.core.AbstractInput.readText(AbstractInput.kt:468)
	at io.ktor.utils.io.core.AbstractInput.readText$default(AbstractInput.kt:465)
	at io.ktor.http.cio.CIOMultipartDataBase.partToData(CIOMultipartData.kt:127)
	at io.ktor.http.cio.CIOMultipartDataBase$partToData$1.invokeSuspend(CIOMultipartData.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:844)

InvalidMutabilityException on macOS when API call fails

https://youtrack.jetbrains.com/issue/KTOR-429

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1759

Ktor 1.3.2 Kotlin 1.3.71 Coroutines 1.3.5 Engine: Curl

Running native on macOS. When doing an API call that return an error, like 500 (internal server errror), ktor throws the following exception:

Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlin.collections.HashMap@d1d12608
        at 0   pos-ecr.kexe                        0x0000000109c43237 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37)
        at 1   pos-ecr.kexe                        0x0000000109c3c4f5 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
        at 2   pos-ecr.kexe                        0x0000000109c3c695 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
        at 3   pos-ecr.kexe                        0x0000000109c71255 kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException + 85 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:22:60)
        at 4   pos-ecr.kexe                        0x0000000109c72ad8 ThrowInvalidMutabilityException + 680 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:90:11)
        at 5   pos-ecr.kexe                        0x0000000109d63ac8 MutationCheck + 104
        at 6   pos-ecr.kexe                        0x0000000109c52a1d kfun:kotlin.collections.HashMap.<set-length>#internal + 77 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/collections/HashMap.kt:16:17)
        at 7   pos-ecr.kexe                        0x0000000109c5756d kfun:kotlin.collections.HashMap.addKey$stdlib(K)kotlin.Int + 1181 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/collections/HashMap.kt:279:36)
        at 8   pos-ecr.kexe                        0x0000000109c53b5d kfun:kotlin.collections.HashMap.put(K;V)V? + 317 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/collections/HashMap.kt:63:21)
        at 9   pos-ecr.kexe                        0x0000000109eacebc kfun:io.ktor.util.AttributesNative.put#internal + 396 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/posix/src/io/ktor/util/AttributesNative.kt:21:0)
        at 10  pos-ecr.kexe                        0x0000000109f16c7f kfun:io.ktor.client.features.$addDefaultResponseValidation$lambda-1$lambda-0COROUTINE$13.invokeSuspend#internal + 1647 (/Users/teamcity1/teamcity_work/4d622a065c544371/backend.native/build/stdlib/kotlin/util/Standard.kt:83:0)
        at 11  pos-ecr.kexe                        0x0000000109f17380 kfun:io.ktor.client.features.$addDefaultResponseValidation$lambda-1$lambda-0COROUTINE$13.invoke#internal + 256 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/DefaultResponseValidation.kt:23:26)
        at 12  pos-ecr.kexe                        0x0000000109f1f76d kfun:io.ktor.client.features.HttpCallValidator.$validateResponseCOROUTINE$19.invokeSuspend#internal + 1341 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpCallValidator.kt:37:0)
        at 13  pos-ecr.kexe                        0x0000000109f1faa4 kfun:io.ktor.client.features.HttpCallValidator.validateResponse#internal + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpCallValidator.kt:36:21)
        at 14  pos-ecr.kexe                        0x0000000109f22793 kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$18.invokeSuspend#internal + 1139 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpCallValidator.kt:99:29)
        at 15  pos-ecr.kexe                        0x0000000109f23148 kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$18.invoke#internal + 312 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/features/HttpCallValidator.kt:97:61)
        at 16  pos-ecr.kexe                        0x0000000109ea99e2 kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1122 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:207:0)
        at 17  pos-ecr.kexe                        0x0000000109ea903b kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:18:0)
        at 18  pos-ecr.kexe                        0x0000000109ea949a kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal + 474 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:183:16)
        at 19  pos-ecr.kexe                        0x0000000109ea35c9 kfun:io.ktor.util.pipeline.Pipeline.execute(TContext;TSubject)TSubject + 361 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/Pipeline.kt:27:41)
        at 20  pos-ecr.kexe                        0x0000000109f09a6a kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$5.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 2042 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/call/HttpClientCall.kt:75:50)
        at 21  pos-ecr.kexe                        0x0000000109f0a0f4 kfun:io.ktor.client.call.HttpClientCall.receive(io.ktor.client.call.TypeInfo)kotlin.Any + 308 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/call/HttpClientCall.kt:67:13)
        at 22  pos-ecr.kexe                        0x0000000109bdcff0 kfun:com.mobilepay.pos.api.V10PointOfSalesRepository.$createPointOfSaleCOROUTINE$5.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 9296 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/call/HttpClientCall.kt:148:65)
        at 23  pos-ecr.kexe                        0x0000000109c637fc kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 700 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:26:0)
        at 24  pos-ecr.kexe                        0x0000000109ea9ff6 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 918 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:238:18)
        at 25  pos-ecr.kexe                        0x0000000109ea9bcf kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1615 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:194:21)
        at 26  pos-ecr.kexe                        0x0000000109eabd21 kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal + 353 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:144:13)
        at 27  pos-ecr.kexe                        0x0000000109c63add kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 1437 (/Users/teamcity1/teamcity_work/4d622a065c544371/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:26:0)
        at 28  pos-ecr.kexe                        0x0000000109ea9ff6 kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal + 918 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:238:18)
        at 29  pos-ecr.kexe                        0x0000000109ea9bcf kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1615 (/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/pipeline/PipelineContext.kt:194:21)

ktor client does not log request body

https://youtrack.jetbrains.com/issue/KTOR-442

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1729

Ktor Version and Engine Used (client or server and name) ver: 1.3.1, 1.3.2 engine: Apache app: client dependencies:

dependencies {
    implementation(kotlin("stdlib-jdk8"))
    implementation(platform("io.ktor:ktor-bom:1.3.2"))
    implementation("io.ktor:ktor-client-core")
    implementation("io.ktor:ktor-client-logging-jvm")
    implementation("io.ktor:ktor-client-apache")
    implementation("io.ktor:ktor-client-json-jvm")
    implementation("org.slf4j:slf4j-simple:1.7.26")
    implementation("org.apache.httpcomponents:httpasyncclient:4.1.4")
}

Describe the bug Request body is not logged even with LogLevel.ALL

To Reproduce

package com.example

import io.ktor.client.HttpClient
import io.ktor.client.engine.apache.Apache
import io.ktor.client.features.logging.DEFAULT
import io.ktor.client.features.logging.LogLevel
import io.ktor.client.features.logging.Logger
import io.ktor.client.features.logging.Logging
import io.ktor.client.request.post
import io.ktor.util.KtorExperimentalAPI
import kotlinx.coroutines.runBlocking

@KtorExperimentalAPI
fun main() {
    runBlocking {
        val client = HttpClient(Apache) {
            install(Logging) {
                logger = Logger.DEFAULT
                level = LogLevel.ALL
            }
        }
        client.post<String>("http://localhost:8080") {
            body = """{ "test": "testVal" }"""
        }
    }
}

Expected behavior Request body is in the log

Actual result

[main] INFO io.ktor.client.HttpClient - REQUEST: http://localhost:8080/
[main] INFO io.ktor.client.HttpClient - METHOD: HttpMethod(value=POST)
[main] INFO io.ktor.client.HttpClient - COMMON HEADERS
[main] INFO io.ktor.client.HttpClient - -> Accept-Charset: UTF-8
[main] INFO io.ktor.client.HttpClient - -> Accept: */*
[main] INFO io.ktor.client.HttpClient - CONTENT HEADERS
[main] INFO io.ktor.client.HttpClient - BODY Content-Type: text/plain; charset=UTF-8
[main] INFO io.ktor.client.HttpClient - RESPONSE: 200 
---   remaining response stuff skipped for brevity  ---

Ktor CIO client, connect timeout handling

https://youtrack.jetbrains.com/issue/KTOR-446

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1720

Using the following code (full example at https://github.com/rrva/ktor-client-connection-leak)
Run the main method repeatedly. Sometimes, all futures created will not complete . Why? Expected behavior is that all created futures will complete eventually, so Done with all will be printed.

 class Client(val fooServiceUrl: String) {

    val client = HttpClient(CIO) {
        engine {
            requestTimeout = 500
            endpoint.connectTimeout = 100
            endpoint.connectRetryAttempts = 2
        }
        install(JsonFeature) {
            serializer = KotlinxSerializer(
                Json(
                    JsonConfiguration(
                        isLenient = true,
                        ignoreUnknownKeys = true,
                        serializeSpecialFloatingPointValues = true,
                        useArrayPolymorphism = true
                    )
                )
            )
        }
    }

    suspend fun fetchFoo(ids: List<String>): List<Foo> {
            return client.request {
                header("User-Agent", "Foo")
                parameter("input", ids.joinToString(","))
                url("$fooServiceUrl/foo")
            }
    }

}

@Serializable
data class Foo(val id: String)
val client = Client("http://localhost:9090")

fun fetchFooAsync(ids:List<String>): CompletableFuture<List<Foo>> {
    return CoroutineScope(Job() + Dispatchers.IO + MDCContext()).async {
        try {
            client.fetchFoo(ids)
        } catch (e: Throwable) {
            println(e.message)
            listOf<Foo>(Foo("timeout"))
        }
    }.asCompletableFuture()
}


fun main() {
    HttpServer.create(InetSocketAddress(9090), 1000).apply {

        createContext("/foo") { http ->
            http.responseHeaders.add("Content-type", "application/json")
            http.sendResponseHeaders(200, 0)
            PrintWriter(http.responseBody).use { out ->
                out.println("""[{"id":"1"}]""")
            }
        }

        start()
    }
    println("Start")

    val requests = (1..100).map {
        fetchFooAsync(listOf(it.toString()))
    }
    for (request in requests) {
        println("Waiting for coroutine...")
        println(request.get())
        println("Done waiting")
    }

    println("Done with all")

}

Reproduced on 1.3.1

Content-Disposition additional parameters should be inside quotes

Empty body in response using macosx64 target

https://youtrack.jetbrains.com/issue/KTOR-479

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1622

Ktor Version and Engine Used (client or server and name) 1.3.0, client (macosx64) (using curl)

Describe the bug When using macosx64 target in a Multiplatform project I'm getting empty BODY in response

HttpClient: REQUEST: http://api.open-notify.org/astros.json
HttpClient: METHOD: HttpMethod(value=GET)
HttpClient: COMMON HEADERS
HttpClient: -> Accept: application/json
HttpClient: -> Accept-Charset: UTF-8
HttpClient: CONTENT HEADERS
HttpClient: BODY Content-Type: null
HttpClient: BODY START
HttpClient: BODY END

The same code for iOS logs

HttpClient: REQUEST: http://api.open-notify.org/astros.json
HttpClient: METHOD: HttpMethod(value=GET)
HttpClient: COMMON HEADERS
HttpClient: -> Accept: application/json
HttpClient: -> Accept-Charset: UTF-8
HttpClient: CONTENT HEADERS
HttpClient: BODY Content-Type: null
HttpClient: BODY START
HttpClient: BODY END
HttpClient: BODY Content-Type: application/json
HttpClient: BODY START
HttpClient: {"people": [{"craft": "ISS", "name": "Andrew Morgan"}, {"craft": "ISS", "name": "Oleg Skripochka"}, {"craft": "ISS", "name": "Jessica Meir"}], "message": "success", "number": 3}
HttpClient: BODY END

The following is the common code that uses ktor

class PeopleInSpaceApi {
    private val url = "http://api.open-notify.org/astros.json"

    private val client by lazy {
        HttpClient() {
            install(JsonFeature) {
                serializer = KotlinxSerializer(Json(JsonConfiguration(strictMode = false)))
            }
            install(Logging) {
                logger = Logger.DEFAULT
                level = LogLevel.ALL
            }
        }
    }

    suspend fun fetchPeople(): AstroResult {
        return client.get(url) 
    }
}

I'm using following dependencies

        macOSMain.dependencies {
            // Coroutines
            implementation('org.jetbrains.kotlinx:kotlinx-coroutines-core-macosx64') {
                version {
                    strictly '1.3.3-native-mt'
                }
            }

            // Ktor
            implementation "io.ktor:ktor-client-curl:${Versions.ktor}"
            implementation "io.ktor:ktor-client-core-macosx64:${Versions.ktor}"
            implementation "io.ktor:ktor-client-json-macosx64:${Versions.ktor}"
            implementation "io.ktor:ktor-client-logging-macosx64:${Versions.ktor}"
            implementation "io.ktor:ktor-client-serialization-macosx64:${Versions.ktor}"

            // Serialize
            implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-macosx64:${Versions.kotlinxSerialization}"

        }

(kotlinxSerialization = "0.14.0")

kotlinMultiplatform publications miss the gradle metadata

HttpClient can only be used on the main thread for native targets

https://youtrack.jetbrains.com/issue/KTOR-491

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1559

Ktor Version and Engine Used (client or server and name) Ktor client 1.2.6, default platform engine (iOS)

Describe the bug HttpClient can only be used from the main thread. It does not matter on which thread the object is instantiated.

The issue is caused by global declarations that are not annotated properly for Kotlin/Native.

Uncaught Kotlin exception: io.ktor.http.URLParserException: Fail to parse url: Caused by: kotlin.native.IncorrectDereferenceException: Trying to access top level value not marked as @ThreadLocal or @SharedImmutable from non-main thread at 0 test.kexe 0x0000000101edb8e7 kfun:kotlin.Throwable.(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37) at 1 test.kexe 0x0000000101ed4ee5 kfun:kotlin.Exception.(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44) at 2 test.kexe 0x0000000101ed4aa5 kfun:kotlin.RuntimeException.(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44) at 3 test.kexe 0x0000000101f07b55 kfun:kotlin.native.IncorrectDereferenceException.(kotlin.String)kotlin.native.IncorrectDereferenceException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/Runtime.kt:30:36) at 4 test.kexe 0x0000000101f25e99 ThrowIncorrectDereferenceException + 137 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/internal/RuntimeUtils.kt:87:11) at 5 test.kexe 0x0000000102200ed9 CheckIsMainThread + 25 at 6 test.kexe 0x00000001020e16a3 kfun:io.ktor.http.<get-URL_ALPHABET_CHARS>#internal + 35 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:11:9) at 7 test.kexe 0x00000001020e2ecd kfun:io.ktor.http.encodeURLPath@kotlin.String.()kotlin.String + 717 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:59:0) ...

I am using the CoroutineWorker library from Autodesk.

To Reproduce Steps to reproduce the behavior:

Make a client request from a non-main thread for a native target.

kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlinx.coroutines.StandaloneCoroutine@28a82d8

https://youtrack.jetbrains.com/issue/KTOR-496

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1550

works on android, breaks on ios

code:

fun triggerNetworkCall(){
      GlobalScope.apply {
            launch(Background) {
                try {
                    val data = repository.getSettings()
                } catch (t: Throwable) {
                    Logger.d("ktor", t.message?:"null")
                }
            }
        }
}

 suspend fun getSettings(): String? {
        val response =  apiConfig.getReferralData()
        return response
}

suspend fun getReferralData(): String? {
       val localClient = HttpClient(PlatformHttpClient.httpClientEngine){
            install(JsonFeature)
        }
        return localClient.post {
            url {
                protocol = URLProtocol.HTTPS
                host = "postman-echo.com"
                encodedPath = "post"
            }
            contentType(ContentType.Application.Json)
            body = "{}"
            HttpMethod.Post
        }
}

This post request works in android but fails in iOS with following stack trace:

Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlinx.coroutines.StandaloneCoroutine@1a8b4e8

whereas a get request works on both platforms. Following is the function being used:

suspend fun getReferralData(): String? {
       val localClient = HttpClient(PlatformHttpClient.httpClientEngine){
            install(JsonFeature)
        }

        val address = Url("https://postman-echo.com/get?foo1=bar1&foo2=bar2")
        return localClient.get {
            url {
                url(address.toString())
            }
        }
}

Dispatchers definition: iOS

internal actual val Main: CoroutineDispatcher = NsQueueDispatcher(dispatch_get_main_queue())

internal actual val Background: CoroutineDispatcher = Main


internal class NsQueueDispatcher(
    private val dispatchQueue: dispatch_queue_t
) : CoroutineDispatcher() {
    override fun dispatch(context: CoroutineContext, block: Runnable) {
        dispatch_async(dispatchQueue) {
            block.run()
        }
    }
}

android

internal actual val Main: CoroutineDispatcher = Dispatchers.Main

internal actual val Background: CoroutineDispatcher = Dispatchers.Default

dependencies

commonMain {

            dependencies {
                implementation "org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61"
                implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-common:0.14.0"
                implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-common:1.3.2-1.3.60"
                implementation "co.touchlab:stately:0.9.4"
                implementation "co.touchlab:stately-collections:0.9.4"
                implementation "io.ktor:ktor-client-core:1.2.6"
                implementation "io.ktor:ktor-client-json:1.2.6"
                implementation "io.ktor:ktor-client-logging:1.2.6"
                implementation "io.ktor:ktor-client-serialization:1.2.6"
                implementation "com.github.aakira:napier:1.1.0"

            }
        }

        androidMain {
            dependencies {
                implementation "org.jetbrains.kotlin:kotlin-stdlib:1.3.61"
                implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.14.0"
                implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.2-1.3.60"
                implementation "io.ktor:ktor-client-core-jvm:1.2.6"
                implementation "io.ktor:ktor-client-json-jvm:1.2.6"
                implementation "io.ktor:ktor-client-logging-jvm:1.2.6"
                implementation "io.ktor:ktor-client-serialization-jvm:1.2.6"
                implementation "io.ktor:ktor-client-android:1.2.6"
                implementation "com.github.aakira:napier-android:1.1.0"
            }
        }

        iosMain {
            dependencies {
                implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-native:0.14.0"
                implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-native:1.3.2-1.3.60"

                implementation "io.ktor:ktor-client-core-native:1.2.6"
                implementation "io.ktor:ktor-client-json-native:1.2.6"
                implementation "io.ktor:ktor-client-logging-native:1.2.6"
                implementation "io.ktor:ktor-client-serialization-native:1.2.6"
                implementation "io.ktor:ktor-client-ios:1.2.6"
                implementation "com.github.aakira:napier-ios:1.1.0"
            }
        }

kotlin.native.concurrent.InvalidMutabilityException with 1.3.3-native-mt

https://youtrack.jetbrains.com/issue/KTOR-497

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1543

KTOR native with the new multi threaded coroutines fails with InvalidMutabilityException. coroutines version: 1.3.3-native-mt ktor version: 1.3.0-rc2 Here is the related code:

suspend fun doFetch() {
    HttpClient().use {
        val htmlContent = it.get<String>("https://en.wikipedia.org/wiki/Main_Page")
        println(htmlContent)
    }
}

fun doesNotWork() {
    runBlocking(Dispatchers.Default) { // remove Dispatchers.Default and it works
        doFetch()
    }
}

fun main() {
    doesNotWork()
}

to reproduce, checkout the following project: https://github.com/yigit/ktor-coroutines-bug

run ./gradlew runDELX --stacktrace from command line.

It fails with :

Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.utils.io.ByteChannelNative@c006ba18
        at kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable (0x2d9607)
        at kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception (0x2d3115)
        at kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException (0x2d2cd5)
        at kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException (0x2fff65)
        at ThrowInvalidMutabilityException (0x301673)
        at MutationCheck (0x5a742e)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.<set-waitingForRead>#internal (0x403fed)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$awaitSuspendCOROUTINE$73.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x417d9f)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.awaitSuspend(kotlin.Int)ValueType (0x418244)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$readRemainingSuspendCOROUTINE$60.invokeSuspend#internal (0x40f22c)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.readRemainingSuspend#internal (0x40f5c0)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$readRemainingCOROUTINE$59.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x40e936)
        at kfun:io.ktor.utils.io.ByteChannelSequentialBase.readRemaining(kotlin.Long;kotlin.Int)io.ktor.utils.io.core.ByteReadPacket (0x40ec6e)
        at kfun:io.ktor.utils.io.readRemaining@io.ktor.utils.io.ByteReadChannel.()io.ktor.utils.io.core.ByteReadPacket (0x421354)
        at kfun:io.ktor.client.features.HttpPlainText.Feature.$install$lambda-1COROUTINE$26.invokeSuspend#internal (0x53c9c4)
        at kfun:io.ktor.client.features.HttpPlainText.Feature.$install$lambda-1COROUTINE$26.invoke#internal (0x53d3ef)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d3e3d)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x4d34be)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceedWith#internal (0x4d3697)
        at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$22.invokeSuspend#internal (0x538612)
        at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$22.invoke#internal (0x538baf)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d3e3d)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x4d34be)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal (0x4d38c1)
        at kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any (0x4ce690)
        at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$12.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x5205d3)
        at kfun:io.ktor.client.call.HttpClientCall.receive(io.ktor.client.call.TypeInfo)kotlin.Any (0x520c06)
        at kfun:$doFetchCOROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x565423)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4968)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
        at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
        at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
        at kfun:kotlinx.coroutines.DispatchedTask.run() (0x3e2fca)
        at kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType (0x3bb507)
        at kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) (0x3f665c)
        at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal (0x3fe271)
        at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$80.invoke#internal (0x3fe3db)
        at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$80.$<bridge-UNN>invoke()#internal (0x3fe43b)
        at WorkerLaunchpad (0x30129c)
        at _ZN6Worker19processQueueElementEb (0x5ac1c9)
        at _ZN12_GLOBAL__N_113workerRoutineEPv (0x5aca2d)
        at  (0x7f54c8681c73)
        at clone (0x7f54c83b3def)
        at  ((nil))

Error Ktor running on background thread on iOS

https://youtrack.jetbrains.com/issue/KTOR-499

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1538

Ktor Version and Engine Used (client or server and name) Ktor client version 1.3.0-rc2 build.gradle:

ext.ktor_version = '1.3.0-rc2'
ext.kotlin_coroutines_version = '1.3.3-native-mt'

sourceSets {
        commonMain {
            dependencies {
                implementation kotlin('stdlib-common')
                implementation ("io.ktor:ktor-client-core:$ktor_version") {
                    exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
                }
                implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-common:$kotlin_coroutines_version"
            }
        }
        iosMain {
            dependencies {
                implementation ("io.ktor:ktor-client-ios:$ktor_version") {
                    exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-native"
                    exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
                }
                implementation ("io.ktor:ktor-client-core-native:$ktor_version") {
                    exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-native"
                    exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
                }
                implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-native:$kotlin_coroutines_version"
            }
        }
    }

Describe the bug Executing an API request on the background thread results in kotlin.native.concurrent.InvalidMutabilityException Full stacktrace

kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@b5b748
        at 0   app                                 0x0000000105688a87 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87
        at 1   app                                 0x0000000105682215 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85
        at 2   app                                 0x0000000105681dd5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85
        at 3   app                                 0x00000001056b6005 kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException + 85
        at 4   app                                 0x00000001056b77d8 ThrowInvalidMutabilityException + 680
        at 5   app                                 0x00000001059a8c58 MutationCheck + 104
        at 6   app                                 0x00000001058c9400 kfun:io.ktor.util.pipeline.Pipeline.<set-interceptors>#internal + 96
        at 7   app                                 0x00000001058c98f5 kfun:io.ktor.util.pipeline.Pipeline.notSharedInterceptorsList#internal + 85
        at 8   app                                 0x00000001058c8a8a kfun:io.ktor.util.pipeline.Pipeline.cacheInterceptors#internal + 650
        at 9   app                                 0x00000001058c97a4 kfun:io.ktor.util.pipeline.Pipeline.sharedInterceptorsList#internal + 340
        at 10  app                                 0x00000001058c5f57 kfun:io.ktor.util.pipeline.Pipeline.createContext$ktor-utils(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)io.ktor.util.pipeline.PipelineExecutor<#GENERIC_kotlin.Any> + 263
        at 11  app                                 0x00000001058c5d71 kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any + 273
        at 12  app                                 0x000000010590797a kfun:io.ktor.client.HttpClient.$executeCOROUTINE$6.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 666
        at 13  app                                 0x0000000105907cc4 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 308
        at 14  app                                 0x000000010593bb7c kfun:io.ktor.client.statement.HttpStatement.$executeUnsafeCOROUTINE$35.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 956
        at 15  app                                 0x000000010593be8b kfun:io.ktor.client.statement.HttpStatement.executeUnsafe$ktor-client-core()io.ktor.client.statement.HttpResponse + 235
        at 16  app                                 0x000000010594b557 kfun:sample.ClientApi.$getQuote$lambda-0COROUTINE$0.invokeSuspend#internal + 6775
        at 17  app                                 0x00000001056a9ec8 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 712
        at 18  app                                 0x00000001057ca07c kfun:kotlinx.coroutines.DispatchedTask.run() + 2780
        at 19  app                                 0x000000010577fa18 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
        at 20  app                                 0x00000001057e3e31 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
        at 21  app                                 0x00000001057eb412 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
        at 22  app                                 0x00000001057eb5fb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$71.invoke#internal + 59
        at 23  app                                 0x00000001057eb65b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$71.$<bridge-UNN>invoke()#internal + 59
        at 24  app                                 0x00000001056b73a1 WorkerLaunchpad + 177
        at 25  app                                 0x00000001059b1909 _ZN6Worker19processQueueElementEb + 2569
        at 26  app                                 0x00000001059b1eb6 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 27  libsystem_pthread.dylib             0x00007fff51bfe2eb _pthread_body + 126
        at 28  libsystem_pthread.dylib             0x00007fff51c01249 _pthread_start + 66
        at 29  libsystem_pthread.dylib             0x00007fff51bfd40d thread_start + 13

To Reproduce Steps to reproduce the behavior:

Clone sample app from https://github.com/aleksey-chugaev/ktor-mt-app
Run iosApp in emulator
See error

Expected behavior Should successfully execute the request.

Url(…) doesn't parse non-HTTP-like URL strings correctly

Code autoreload not working for me

MPP JS client: Can't resolve 'text-encoding'

kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@545e18f8

https://youtrack.jetbrains.com/issue/KTOR-693

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/887

Ktor Version

implementation("io.ktor:ktor-client-ios:1.1.1")

Ktor Engine Used(client or server and name)

Client

JVM Version, Operating System and Relevant Context

MBP 2017 JVM 1.8

Feedback

This exception was printed out,and the test can't print out the expected result

    suspend fun getDemoJSONDataFromNet(): String {


        val result = client.get<String>("https://en.wikipedia.org/wiki/Main_Page")

        Logger.d(tag, "the return data is $result")

        return result
    }

fun getDataIos(callBack: (result: String) -> Unit) {
    runBlocking {
       val job = async {
            val result = HttpUtil.getDemoJSONDataFromNet()
            Logger.d("SimpleIos","result value is $result")
            callBack(result)
        }
        job.await()
    }
}

    @Test
    fun testGetData() {
        getDataIos {
            println(it)
        }
    }

Exception Log:

Instances of kotlin.Error, kotlin.RuntimeException and subclasses aren't propagated from Kotlin to Objective-C/Swift.
Other exceptions can be propagated as NSError if method has or inherits @Throws annotation.
Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@2ef2a08
        at 0   app                                 0x000000010dba2896 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 70
        at 1   app                                 0x000000010dba27b6 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 70
        at 2   app                                 0x000000010dba40e6 kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException + 70
        at 3   app                                 0x000000010dc3c6b8 ThrowInvalidMutabilityException + 280
        at 4   app                                 0x000000010dc5ee48 MutationCheck + 24
        at 5   app                                 0x000000010dce8a83 kfun:io.ktor.util.pipeline.Pipeline.<set-interceptors>#internal + 67
        at 6   app                                 0x000000010dce86a6 kfun:io.ktor.util.pipeline.Pipeline.notSharedInterceptorsList#internal + 70
        at 7   app                                 0x000000010dce84d6 kfun:io.ktor.util.pipeline.Pipeline.cacheInterceptors#internal + 710
        at 8   app                                 0x000000010dce6292 kfun:io.ktor.util.pipeline.Pipeline.sharedInterceptorsList#internal + 98
        at 9   app                                 0x000000010dce61ca kfun:io.ktor.util.pipeline.Pipeline.createContext$ktor-utils(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)io.ktor.util.pipeline.PipelineExecutor<#GENERIC_kotlin.Any> + 90
        at 10  app                                 0x000000010dce60f0 kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any + 96
        at 11  app                                 0x000000010dd313a8 kfun:io.ktor.client.HttpClient.$execute$COROUTINE$1.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 296
        at 12  app                                 0x000000010dd31522 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 130
        at 13  app                                 0x000000010dd32696 kfun:io.ktor.client.call.$call$COROUTINE$3.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 758
        at 14  app                                 0x000000010dd32852 kfun:io.ktor.client.call.call@io.ktor.client.HttpClient.(kotlin.coroutines.SuspendFunction1<io.ktor.client.request.HttpRequestBuilder,kotlin.Unit>)io.ktor.client.call.HttpClientCall + 130
        at 15  app                                 0x000000010dd330eb kfun:io.ktor.client.call.call@io.ktor.client.HttpClient.(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 123
        at 16  app                                 0x000000010db97f3f kfun:sample.HttpUtil.$getDemoJSONDataFromNet$COROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 1551
        at 17  app                                 0x000000010db97821 kfun:sample.HttpUtil.getDemoJSONDataFromNet()kotlin.String + 113
        at 18  app                                 0x000000010dba0de9 kfun:sample.$getDataIos$lambda-1$lambda-0$COROUTINE$3.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 313
        at 19  app                                 0x000000010dbd4801 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 385
        at 20  app                                 0x000000010dca7b89 kfun:kotlinx.coroutines.DispatchedTask.run() + 1129
        at 21  app                                 0x000000010dca92d6 kfun:kotlinx.coroutines.EventLoopImpl.processNextEvent()ValueType + 422
        at 22  app                                 0x000000010dccf7f7 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 263
        at 23  app                                 0x000000010dccf5d7 kfun:kotlinx.coroutines.runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,#GENERIC>)Generic + 983
        at 24  app                                 0x000000010dccfb4c kfun:kotlinx.coroutines.runBlocking$default(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,#GENERIC>;kotlin.Int)Generic + 204
        at 25  app                                 0x000000010dba06e4 kfun:sample.getDataIos(kotlin.Function1<kotlin.String,kotlin.Unit>) + 100
        at 26  app                                 0x000000010dba0618 kfun:sample.main(kotlin.Array<kotlin.String>) + 216
        at 27  iosApp                              0x000000010d897774 $S6iosApp14ViewControllerC11viewDidLoadyyFTo + 36
        at 28  UIKitCore                           0x0000000112ea84e1 -[UIViewController loadViewIfRequired] + 1186

When Route.authenticate has multiple configurations (>1) only first configuration is used to authenticate

OAuth feature is incompatible with Dropbox

https://youtrack.jetbrains.com/issue/KTOR-715

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/771

Ktor Version

1.0.0

Ktor Engine Used(client or server and name)

Server: Netty Client: Apache Using OAuth feature

JVM Version, Operating System and Relevant Context

JVM: 10 OS: Mac

Feedback

OAuth2 with Dropbox fails during the "request access token" step. In summary, the OAuth feature includes the state parameter in the request. Dropbox's API does not allow unexpected parameters, and fails the OAuth access token request.

From what I can tell, there is currently no way to prevent the state from being added to the access token request.

Details

Consider the following 3 client-server exchanges. Here the Client is the Ktor server, and the Server is the identity provider (Dropbox, Google, Github, etc).

[Client] -- /authorize -> [Server]
[Client] <- ...?code=... -- [Server]
[Client] -- /access_token -> [Server]

In a typical OAuth2 exchange, the Client sends a state=<nonce> parameter with the request in 1 and the Server responds with state=<nonce> in the redirect in 2. (This state sending is not mandatory).

The OAuth feature is configured to include the state in 3 when the Client requests the access token from the Server. Some APIs such as Github's want the state to be included in the request. Dropbox however, reject requests that include the state.

The following code sets the state on the access token request. https://github.com/ktorio/ktor/blob/85210082c729b12099ad25f509e99b23ccea23b8/ktor-features/ktor-auth/src/io/ktor/auth/OAuth2.kt#L149-L151 In short, if a state was returned from the Server in 2, it will be non-null at this point in the code.

Using a custom `OAuth2StateProvider`

OAuth2ServerSettings allows the settings of a custom OAuth2StateProvider with which we can generate a custom state. I mentioned previously that the state is not mandatory, so maybe we can not generate a state? This is not an option, as the state must not be null. An empty state results in a request that has state=, but state is still there.