Fluid Flow Control

A cylinder in a stream leaves alternating vortices behind it. An airfoil must produce lift without wasting too much energy overcoming drag. A nozzle must point its jet in the right direction. In each case, a controller reads sensors, changes an actuator and watches how the fluid responds. The challenge is to act early enough to help without creating a larger disturbance.

The task brings together 39 simulated cases in two and three dimensions. One jointly trained model must serve all of them. The scientific question is whether experience with one flow can help with another, even when their sensors, actuators and goals differ. Separate specialist controllers provide useful comparisons, but a collection of specialists does not meet the shared-model goal.

Training data come from interactions with a fluid simulator: a sensor reading, an action and the resulting motion. All supplied cases can be used during training, so this tests learning across a known suite rather than discovering entirely new physics at evaluation time.

A few sensors reveal only part of a moving flow. Remembering earlier measurements may help a controller infer whether a vortex is growing or passing by. The interactive oscillator illustrates this feedback problem; it is a teaching model, not a measured fluid field.

Lower drag is useful for a cylinder, while an airfoil benefits from higher lift relative to drag. Gust control asks how far aerodynamic forces depart from their targets; other cases measure jet-angle error or turbulence. These quantities keep their own units and are reported separately.

For many cases, the score uses the final half of the response so that an initial transient does not dominate. For example, if the last three drag measurements are 1.4, 1.2 and 1.0, their mean is 1.2. A low final reading alone does not establish sustained control. Gust and nozzle cases instead include the full response.

The starter places each case’s sensor values into a common input, marks which entries contain real measurements and identifies the current task. This lets one network handle different sensor layouts without confusing an absent sensor with a measured zero.

It learns with proximal policy optimization, or PPO: try actions, estimate whether they helped, then cautiously increase the chance of useful actions. A second network estimates future reward during training. At evaluation, the trained controller produces commands within each actuator’s physical limits. Training cases in long separate phases can cause forgetting, making the order and balance of experience important.

The specialist comparisons show both possibilities. On the two-dimensional cylinder at Reynolds number 200, control reduces drag by about 16%. Reynolds number describes the balance between inertia and viscosity. On the three-dimensional cylinder at Reynolds number 3900, the recorded controller instead increases drag by about 9%.

These are results for separately trained specialists, not evidence that the shared starter achieves the same performance. Different training budgets also prevent a clean ranking of learning algorithms. The useful lesson is to inspect improvements and regressions across cases, rather than trust a rising training reward.

A promising experiment would alternate cases more frequently while keeping their total training experience fixed. Does this preserve earlier skills? Another would add memory to the controller to test whether incomplete sensor coverage is the bottleneck. Both questions need case-by-case physical results and actuation costs: suppressing a wake by spending excessive control energy may be a poor trade.