Atelier Ryza: Ever Stuttering and the Stupid Workaround

Like every other Atelier game, the PC port leaves much to be desired, especially in terms of input. A lack of mouse support, along with only direct mappings from keys to controller inputs (which are non-intuitive at best) takes a bit of time to get used to.

As of version 1.03, the Steam version of Atelier Ryza has some stuttering issues. About every 1-2 seconds, you will probably experience some frame skips. Accompanied by these frame skips are the physics engine glitching out, which probably means the physics are tied to the rendering loop. Fantastic.

You might be wondering: how did something so obviously broken make it past QA? Well, wonder no more.

Every 1-2 seconds, the game polls the XINPUT library for controller input. This poll either has a timeout, or has a long handler in the event of failure. The input polling is probably done in the rendering loop, which causes the frame stutter. The people at Gust probably did their QA with a controller plugged in (they would probably get an RSI from testing with a keyboard all day), and the glaringly obvious problem slipped into release.

How can you work around this? The easiest way would be to get a controller. If you don’t have one, then you can use vJoy to emulate a joystick.

And no, I don’t mean “use vJoy as a controller to play the game” (which would require something like x360ce to send inputs to the driver anyways). I mean “install vJoy ‘‘literally’’ just so the game thinks you have a controller plugged in”, so the XINPUT polling doesn’t fail. You’ll still be playing with the keyboard.

It’s $(current year) folks.

Supporting Legacy of Lunatic Kingdom w/ Twinject

twinject (touhou windows injector) is an automated player for the bullet hell games from the Touhou Project. Instead of using computer vision techniques to extract data from the game, a DLL is injected and the relevant game data is directly extracted through trampolined functions.

For the unaware, bullet hells are a category of Shoot ‘em Up video games where the player controls a ship, which must dodge large numbers of obstacles and destroy large numbers of enemies. In the demonstration video, the fast moving projectiles are the obstacles which must be dodged - if the player hits any of these projectiles they die immediately. The player itself also fires projectiles, which damage enemies. The player is the little red sprite near the bottom of the screen. Props to ZUN for making games that people still play 20 years later.

If you want to see a prototype, you can check out the video demonstration below.

Video Demonstration (Legacy of Lunatic Kingdom)

Theoretically, Twinject supports every Windows game in the Touhou Project. In order to support a game, four main components must be implemented, which I will briefly detail below.

Components

Graphical Control

Twinject requires control over the game graphics in order to draw debugging information which is useful in the development of the other components.

All mainline Touhou games can use Direct3D9 for their graphics (older games through the dxd8->dxd9 patch). Thus, this component is not game specific and only needs to be implemented once.

This component is implemented as a object wrapper delivered to the game via trampolining Direct3DCreate9.

Input Control

Twnject requires control over the game input in order to control the player.

All mainline Touhou games can use DirectInput8 for their input (configurable with config.exe). Thus, this component is not game specific and only needs to be implemented once.

This component is implemented as a object wrapper delivered to the game via trampolining DirectInput8Create.

Specific Data Processor

Twinject requires game data in order to be aware of the player’s environment. This includes,

• Player location, velocity
• Bullet locations, velocities
• Powerup locations, velocities
• Enemy locations, velocities
• Laser functions

The game engines for each Touhou game vary wildly - this component is game specific and must be implemented for each game.

This component is usually implemented as a function hook.

Algorithm

Twinject requires an algorithm to turn game data into player inputs. The following algorithms are currently supported:

• Method of Virtual Potential Fields
• Method of Velocity Obstacles
• Method of Vector-based Velocity Projection

I will explain these algorithms in detail in their own post.

Implementation

Twinject provides a layer of abstraction between components, allowing new components to be easily created and connected together to create a functional automated player for a certain game.

Implementation of game-specific components is usually quite involved, and requires reverse-engineering the game. In this article, I will detail the required procedure to implement a game-specific component for Legacy of Lunatic Kingdom (LoLK).

LoLK Specific Data Processor

I will detail the specific component required for processing data in LoLK.

Collidable Locations

The player requires the locations and size of every collidable entity, so that it may dodge them. This includes the player, bullets, lasers, etc. Collidable entities may be demarcated into two types: function-based, and point-based.

The player and bullets are point-based, and are defined as a point with a radius (LoLK uses hit-circles).

Lasers are function-based, and are defined as a curve with a restricted domain.

For now, we will only talk about point-based collidable entities (PBCE).

Obtaining Data

We want to trampoline a function which processes PBCEs once per frame. It would make sense for the game to have a function that checks for collisions between two PBCE.

This website details many memory patches that can be used to modify the functionality of Touhou games. An interesting one is

;無敵
00456540-C3     // set byte at 00456540 to C3

which renders the player invincible.

Taking a look in a disassembler (IDA), we can see that setting that byte causes a function to return immediately when called.

A disassembly in IDA, showing the function that the patch applies to, and its effect. (note that any variable names and comments were added in myself to help me reverse-engineer the game, and were not present beforehand)

Setting a breakpoint in Ollydbg, we can see that this function is called after the player hits an obstacle. This function then carries out the death routine (reduce lives, set invincibility for a while, put player at origin etc.)

Checking cross references to this function, we find the following function:

A rough decompilation of the function, with variable names annotated based on intuition

We can see that this function checks for player collision with PBCEs using the square Euclidean distance check for circle collision. If the PBCEs collide, then it calls the death function we found earlier.

We can write a wrapper function that will extract the relevant data for Twinject to use:

Hook function for collision check function

This wrapper function will be part of the component. Unfortunately the function is usercall, so we must write our own prolog and epilog code with inline assembly.

A Digital Will

Here’s a rather morbid topic. Let’s say you want some sort of “digital will” - things you want completed digitally after you are dead. Why not appoint a human executor? Because humans are fallible, and you don’t know whether or not your testament will be carried out as originally planned.

Dead Man’s Switch (DMS) [#n7bb4d76]

A proposed solution, which is not new, is the concept of a dead man’s switch implemented in software. Before I talk about digital dead man’s switches, it would be helpful to know what a dead man’s switch is.

Let’s say that you are a train conductor, speeding down a rail at 30 mph. From a far distance, you see a bus stuck on the tracks - just far enough so that if you engaged the brakes, they would be spared. Suddenly, you have a heart attack and die. The train doesn’t know you’re dead - it will keep barreling along and plough right through them (there are definitely more fail-safes present with modern train systems, just bear with me).

The problem with this system is that it is not “fail-safe” - that is, when the unexpected occurs, it does not naturally tend to a safe state. How would you solve problem? Take a bit to think about it.

One solution is a physical dead man’s switch - something that would be able to tell when the conductor is unresponsive, and stop the train immediately (i.e. apply the brakes) in response. An example implementation is a foot pedal that requires significant pressure to keep pressed. If the conductor were to lose consciousness, they would probably not be able to keep the pedal down, which would trigger the fail-safe and bring the system to a safe state (i.e. not moving).

Digital Dead Man’s Switch (DDMS)

The same idea applies with a digital dead man’s switch, where we want a system to return to a safe state (or deadly state, depending on the situation) once a certain set of conditions are met (usually operator incapacitation or death). For example, the following situations may warrant the use of a digital dead man’s switch:

• Wiping master keys to encrypted data, making data irrecoverable -Releasing decryption keys to encrypted data already publically known -Delivering information to certain individuals such as account passwords, banking information, etc. -Archiving/deleting social media accounts

The specific type of digital dead man’s switch I will talk about for the rest of this article are those that react to inability of the operator to make judgement and act on decisions. This criteria covers death and incapacitation as well. We will call this requirement the “trigger criteria”.

In response to the trigger criteria, the DMS will carry out a predefined action, which we will call the “trigger action”.

In order to talk about a digital dead man’s switch, we can separate a digital dead man’s switch into two components, detection (regarding trigger criteria) and reaction (regarding trigger action). Additionally, I will also talk about the concept of a trusted host, including distributivity.

Detection

Before we carry out the trigger action, we must know if the trigger criteria has been met. Of course, there is no way for software to directly know if your trigger criteria has been satisfied (especially if it is more complex, like the one I mentioned earlier). Thus, we introduce the idea of a “criteria hint”. The presence of a criteria hint is a sign (but not a guarantee) that the trigger criteria has been met.

For the specific trigger criteria I mentioned earlier, there are several criteria hints which may be used. Any combination of the following hints can be used, with the goal being to have the hints accurately represent the trigger criteria. Criteria hints should be used together in a way (e.g. a weighted sum) that most accurately represents the trigger criteria.

Also note that some criteria hints may depend on others. For example, you might only allow Confirmation by Trusted Entity after Failure to Execute Periodic Action. You want low confidence hints to depend on high confidence hints.

Failure to Execute Periodic Action (FtEPA)

The operator must periodically execute a predefined action that only the operator may perform. Failure to execute the action within a given timeframe constitutes presence of this hint.

An example: the operator must respond to a challenge issued by a trusted server (hosting the DDMS) periodically. The challenge may only be completed with knowledge that the operator is in the sole possession of.

It is important for the predefined action to require the judgement/decision making abilities of the operator, as that is what we are trying to verify. Thus, it must depend on information only the operator knows.

Since this hint depends on the operator, it may be given a high level of confidence. Other hints may depend on this one, since it has a high level of confidence.

Confirmation by Medical Technology (CbMT)

The operator’s well being is continuously verified via medical technology (e.g. vital signs monitor). Failure to confirm well being via medical technology constitutes presence of this hint.

An example: the operator has a heart rate monitoring watch, which will report to a trusted server (hosting the DDMS) when the heart rate is no longer present.

Note that depending on the technology used, this hint may be easy to circumvent. Thus, it should be given a low level of confidence.

Confirmation by Trusted Individual/Organization/Entity (CbTE)

The operator appoints trusted entity(s) that will use their own criteria hints to detect the trigger criteria. A threshold on the number of trusted entities required to constitute presence of the criteria hint may be used, depending on whether the system should be fail-easy or fail-hard.

A bad example: the operator gives their spouse a button which when pressed, will report to a trusted server (hosting the DDMS).

A better example: the operator gives their spouse a phone application that when explicitly utilized, will display a challenge issued by a trusted server (hosting the DDMS). The challenge response will be reported to the trusted server.

It is important for the spouse to be the only entity that can solve the challenge.

The second example is more secure, as it requires the identity of the trusted entity to be verified before they are allowed to exercise their permission as a criteria hint. Trusting an entity without verification is equivalent to trusting EVERYONE.

The level of confidence given to this hint should be related to how much the operator trusts the trusted entity. You may want to only activate this hint if a hint with a higher level of confidence is present (e.g. FtEPA).

Pre-criteria Explicit Self Confirmation (PcESC)

The operator possesses a method to forcefully satisfy the trigger criteria (on a timer) as an anticipation that the trigger criteria will be met in the near future.

An example: the operator possesses a phone application that when explicitly utilized, will display a challenge issued by a trusted server (hosting the DDMS). The challenge response will be reported to the trusted server. The operator is about to undergo a surgical operation where they only have a 10% chance of survival. They trigger the PcESC criteria hint before the operation on a timer, such that if they do survive, they may cancel the hint.

It is important for the operator to be the only entity that can solve the challenge.

This hint should have a high level of confidence, since it depends on the operator.

You may find that the example may be similar to the one used for confirmation by trusted entity (CbTE). This is because they are practically the same, except for two things.

Anticipation

The criteria hints used by trusted entities in CbTE should only be present when your trigger criteria is satisfied.

However, the PcESC hint may be present before the trigger criteria is satisfied. It is the operator that anticipates (using their judgement) that the trigger criteria may be met.

Note that anticipation may also be used in CbTE as well, but the next point issues why that may be a bad idea.

Trust

The only entity that the operator ultimately trusts is the operator themselves. It may be the case that the operator is the only entity that may be trusted with the ability to anticipate meeting the trigger criteria.

Ultimately, it depends on the specific implementation.

Reaction

Now that we have defined the trigger criteria, we want to react accordingly with the trigger action. There is not that much theory regarding reaction, except for a couple key points:

Reaction Intensity

You may only want certain trigger actions to be activated depending on how confident you are in your criteria hints. For example, you might not want your hard drive wiped if just a friend said you were dead, because the action is irreversible(I’ll talk about that more later). However, some safer actions can be taken even if you aren’t confident in your criteria hints, such as locking your computer or phone, or wiping your browser history.

Reversibility

Actions may be be either reversible, irreversible, or somewhere in-between (writing that sentence down didn’t really add anything).

You may want to only carry out irreversible actions (i.e. wiping a disk) if you are extremely confident in your criteria hints.

Reversible actions may be carried out without less thought, as there is a lower consequence when they are incorrect.

Testing

Trigger actions may fail. It is important to test them periodically to make sure they still work. It is also important to be able to detect whether or not they have failed, and have a backup trigger action in case.

Examples

The following are examples of some trigger actions:

• Wipe encryption keys
• Wipe hard disk
• Send message to individuals
• Archive/delete social media

Trusted Host [#ja669901]

You may have noticed that I talked a lot about a trusted server while talking about criteria hints. This is where the majority of the trust in a DDMS is placed.

Trusted Server vs. Trusted Individual

Why is a trusted server safer than a trusted individual?

People change over time. They are not reliable. You may trust them now, but you cannot say that you will 5 years from now. However, they may know better than you, and be able to carry out your task with their judgement.

A computer will carry out your task (without interference) without question, at the downside that they lack judgement regarding how to carry out your task in the best way.

Guaranteeing Trust

It is important that the server you are using for your DDMS is completely under your control. No AWS instances, Google Compute Engine instances. It is important that your server constantly has power. It should also have tamper detection (you can even add this as a criteria hint).

Distributivity

If your DDMS server fails, the entire system fails. It is important to have multiple DDMS servers in several locations, that can detect whether a fellow server has failed (you can even add this as a criteria hint).

Additional Concerns

Reversal

It should not be possible to reverse the “triggered” state. Once the DDMS has been triggered, there should be no way to “cancel” it, if you are confident in your criteria hints. Of course that may not be possible in reality, so adding a cancel operation that requires the operator may be important.

Conclusion

I will update this article with any new information I gain periodically. Please let me know if I have made any mistakes, if anything is unclear, or if you have ideas you want to add.

Discovering a Strange Signal

I got my hands on a software defined radio recently. It’s really interesting to play around with – from things like receiving aircraft positional data, to municipal trunked radio. I spent a good chunk of time just scrolling through the electromagnetic spectrum, hoping that I’d come upon something interesting. I got a lot more than I asked for, with 915.519652MHz.

Ghost in the Spectrometer

The first thing that caught my eye was not the gigantic FM transmission in the middle, but all those tiny little blips. Most likely, they’re just smart meters beeping away. Eventually, that FM waveform started to call out to me. I thought it was weird for an FM broadcast to be happening at this wavelength, because it is usually just radio silence (excluding the smart meters).

After listening to it, I heard a bizarre mixture of noise, gunshots, explosions and every single other disconcerting sound you can think of. I wish I made a recording, but the transmission abruptly stopped before I could do so. The sound was somewhat periodic, with strange distortions happening at an interval. I, for the life of me, could not figure out what could be causing such a transmission at this frequency.

Here’s a screenshot from a brief period of silence. For you experienced radio theorists out there, it might be clear to you what’s causing this pattern. But I have absolutely no clue. From this picture, multiple spikes of a decreasing amplitude can be seen around around a larger peak. To me, this kind of looks like some inadvertent transmission caused by some faulty hardware, especially since it was broadcasting on some fractional frequency.

I decided to leave it at that, as a mystery that would never be solved. Alas, I could not have such comfort. The transmission returned at a similar time the next day, to my confusion. This time, it was even more bizarre.

The Second Coming

Recording 1
Recording 2

Here are two recordings I made of the transmission. I honestly had no idea what to think at this point. My mind could not rationalize why a Space Jam remix of Renai Circulation would be broadcasted at 915MHz. A while later, I started hearing a Film Theory video on the Emoji Movie. Of course, this latest broadcast helped me understand a few more things about this transmission.

Analysis

Some facts I collected:

• The broadcasts are temporal. They take place sometime around the day.
• They rapidly switch between different sound clips.
• Most of these clips sound like YouTube videos.
• Sometimes, the broadcast is completely silent. However, there is still a waveform shown, with multiple decreasing peaks.
• Sometimes, the broadcast is completely silent, with no waveform.
• The broadcast is on a strange frequency, and doesn’t snap to a nice number.
• The audio would constantly jump around backwords and forwards.

This made me start to think. What is causing this broadcast? Since YouTube videos were being played at a random pacing, I deduced that someone must have been casually watching YouTube videos somehow. The audio jumping around implied this strongly, perhaps they missed something and jumped backwards in the video to hear it again. Somehow, the way they were watching the video had to have caused radio emissions.

Theory 1: The RF Audio System

The first idea I had is that someone might be using some RF solution to broadcast audio from their computer to some audio system or television. This would explain why YouTube videos were playing, rapidly switching as a person navigated videos. The waveform being present while no sound was playing would represent the sound system being on while nothing was being player, and the waveform being non-present would represent the sound system being off. However, this didn’t explain a few things. For one, I have never heard of the use of the 33cm band for audio systems before, especially some irrational frequency. It also didn’t explain all the interference present around the signal, with multiple copies of the original but at progressively quieter intervals.

Theory 2: Spurious Transmissions

Another idea I had is that someone might be using some FCC violating speakers/sound system, that emits the audio as FM radio as a byproduct. I’ve heard of speaker systems receiving radio transmissions but not the other way around, so the idea doesn’t seem ridiculous to me. This would explain the irrational frequency, since no one would have designed the equipment to broadcast at that strange frequency. This theory also explains the multiple diminished copies of the original signal, which may be a result of multiple resonances that lead to transmission. But, who knows?

Theory 3: I’m completely wrong

I still have no clue, so I would not be surprised if all my conclusions were wrong. If you know what it possibly could be, please tell me so I can put this to rest.

But what about the gunshots and explosions?

Oh yeah, I almost forgot about those. Taking into account the computer-connected-to-speakers-or-sound-system theory, it was probably someone playing a video game.

Anyways, I’m kinda tired of talking about this so I’ll leave it at that. Cheers.