mastodon.world is one of the many independent Mastodon servers you can use to participate in the fediverse.
Generic Mastodon server for anyone to use.

Server stats:

9.6K
active users

#async

10 posts9 participants2 posts today

Укрощаем асинхронный код с помощью async/await

Привет! Меня зовут Андрей Максимкин, я iOS-разработчик в hh. Мы в команде активно используем async/await подход при написании нового кода, а также активно применяем при переписывании старого. В процессе работы сталкивались с некоторыми интересными и не самыми очевидными моментами — их и рассмотрим в статье. Работа с различными потоками — очень важная часть разработки мобильных приложений под iOS. Грамотное распределение нагрузки положительно влияет на скорость работы приложения, а значит, и на пользовательский опыт. До Swift 5.5 для работы с многопоточностью в основном использовали фреймворки GCD и NSOperation. Начиная с версии Swift 5.5 стал доступен функционал async/await. В статье мы кратко расскажем о базовых принципах данного подхода и сделаем акцент на проблемах и особенностях, которые необходимо знать при написании кода. Информация будет полезна тем, кто уже знаком с функционалом async/await, а некоторые примеры могут быть интересны и более продвинутым разработчикам. Поехали!

habr.com/ru/companies/hh/artic

ХабрУкрощаем асинхронный код с помощью async/awaitПривет! Меня зовут Андрей Максимкин, я iOS-разработчик в hh. Мы в команде активно используем async/await подход при написании нового кода, а также активно применяем при переписывании старого. В...
Continued thread

digging a bit deeper to further improve this #async / #await implementation:

Keeping all the #stacks mapped certainly makes sense to avoid excessive #mmap overhead ... but on *some* (most?) systems (I checked #FreeBSD and #Linux docs for now), we can let the kernel know when the mapping just contains worthless scratch (which will be the case after a thread job finishes until another thread job claims the same stack). Then, if there's a need to free pages, the kernel can avoid pointlessly writing that garbage to #swap. 😎

github.com/Zirias/poser/commit

Revisiting #async / #await in #POSIX C, trying to "add some #security" 🙈

Recap: Consider a classic #reactor-style service in C with a #threadpool attached to run the individual request handlers. When such a handler needs to do some I/O, it'll have to wait for its completion, and doing so is kind of straight forward by just blocking the worker thread executing the job until whatever I/O was needed completes.

Now, blocking a thread is never a great thing to do and I recently tooted about an interesting alternative I found: Make use of the (unfortunately deprecated) POSIX user context switching to enable releasing the worker thread while waiting. In a nutshell, you create a context with #makecontext that has its own private #stack, and then you can use #swapcontext to get off the thread, and later again to get back on the thread. A minor issue is: It must be the *same* thread ... so you might have to wait until it completes something else before you can resume your job. But then, that's probably okayish, you can make sure in your job scheduling to only use worker threads with awaited tasks attached when no other thread is available.

In my first implementation, I just used #malloc to create a 64kiB private stack for each thread job. That's perfectly fine if you can guarantee your job will never consume more stack space, AND it won't have any vulnerabilities allowing some attacker to mess with the stack. But in practice, especially for a library offering this async/await implementation, it's nothing but a wild #CVE generator.

So, I now improved on that:

* Allocate a much larger stack of now 2MiB. That alone makes issues at least less likely. And on a sane modern OS, we can still assume pages will only be mapped "on demand".
* Only allocate the stack directly before running the thread job, and delegate allocation to some internal "stack manager" that keeps track of all allocated stacks and reuses them, only freeing them on exit. This should avoid most of the allocation overhead.
* If MAP_ANON / MAP_ANONYMOUS is available, use #mmap for allocating the stack. That at least gives a chance to stay away from other allocations ....
* But finally, if MAP_STACK is available, use this flag! From my research, #FreeBSD, #OpenBSD and #NetBSD will for example make sure there's at least one "guard page" below a stack mapped with this flag, so a stack overflow consistently takes the SIGSEGV emergency exit 😆. #Linux knows this flag as well, but doesn't seem to implement such protection at this time ... 🤔

A couple of weeks ago I asked the Fediverse for recommendations about doing async I/O in a Python application which *also* had a REPL-like interface. I got quite a few useful responses, which ended up leading me to Python Prompt Toolkit.

Here is the result: a single-file script with embedded dependencies (thanks to PEP 723 and uv) which opens a serial port, asynchronously writes data to it and reads data from it, all while displaying a prompt and accepting input from the user.

forge.km6g.us/kpfleming/muart-

None of the program's actual functionality is present yet, but having the scaffolding in place makes me happy.

Summary card of repository kpfleming/muart-failsafe
KM6G Software Forgemuart-failsafe/prototest/prototest.py at 9179da2f6acc30dbead5e54a2de2184b6fe1c7f8muart-failsafe - Failsafe hardware for use with the MUART project

So, question for the #Android #Kotlin crowd: I have a process that is async (upload of a file), and that needs to be run a bunch of times. If I start each upload in a loop, I cannot really cancel or observe, bcs it is retrofit/okhttp who is managing the async stuff and the loop ends fast.
It is entirely possible that every now and again, I have to upload a lot of those files, and not being able to cancel is not acceptable.
So, what are my options?
#BuildInPublic #async #FollowerPower

Today, I implemented the #async / #await pattern (as known from #csharp and meanwhile quite some other languages) ...

... in good old #C! 😎

Well, at least sort of.

* It requires some standard library support, namely #POSIX user context switching with #getcontext and friends, which was deprecated in POSIX-1.2008. But it's still available on many systems, including #FreeBSD, #NetBSD, #Linux (with #glibc). It's NOT available e.g. on #OpenBSD, or Linux with some alternative libc.

* I can't do anything about the basic language syntax, so some boilerplate comes with using it.

* It has some overhead (room for extra stacks, even extra syscalls as getcontext unfortunately also always saves/restores the signal mask)

But then ... async/await in C! 🥳

Here are the docs:
zirias.github.io/poser/api/lat

zirias.github.ioposer: PSC_AsyncTask Class Reference

I finally eliminated the need for a dedicated #thread controlling the pam helper #process in #swad. 🥳

The building block that was still missing from #poser was a way to await some async I/O task performed on the main thread from a worker thread. So I added a class to allow exactly that. The naive implementation just signals the main thread to carry out the requested task and then waits on a #semaphore for completion, which of course blocks the worker thread.

Turns out we can actually do better, reaching similar functionality like e.g. #async / #await in C#: Release the worker thread to do other jobs while waiting. The key to this is user context switching support like offered by #POSIX-1.2001 #getcontext and friends. Unfortunately it was deprecated in POSIX-1.2008 without an obvious replacement (the docs basically say "use threads", which doesn't work for my scenario), but still lots of systems provide it, e.g. #FreeBSD, #NetBSD, #Linux (with #glibc) ...

The posercore lib now offers both implementations, prefering to use user context switching if available. It comes at a price: Every thread job now needs its private stack space (I allocated 64kiB there for now), and of course the switching takes some time as well, but that's very likely better than leaving a task idle waiting. And there's a restriction, resuming must still happen on the same thread that called the "await", so if this thread is currently busy, we have to wait a little bit longer. I still think it's a very nice solution. 😎

In any case, the code for the PAM credential checker module looks much cleaner now (the await "magic" happens on line 174):
github.com/Zirias/swad/blob/57