Add FastUdpController and Esp32S3StripController in AdressableLedStrip namespace. Stability fix for WemosD1StripController and NamedPipeServer (PinOne). - #24
Conversation
- Introduced new `<SetW5500>` XML property to toggle hardware burst pacing on/off (default: true). - Optimized the chunking algorithm to dynamically calculate wait times for remaining chunks (e.g., 0.4ms per chunk) to avoid wasting CPU ticks. - Combined hardware LED latching time and network burst pacing into a single, highly efficient Stopwatch loop at the end of the frame.
This PR extends the WemosD1MPStripController to support up to 16 addressable LED strips, specifically targeting the capabilities of the newer ESP32-S3 microcontrollers used in modern VPinball setups.
Key Changes:
Added a new Enable16ChannelMode boolean property.
Expanded the internal LED strip configuration to support NumberOfLedsStrip11 through NumberOfLedsStrip16.
Overrode UpdateOutputs() to dynamically respect the channel limit.
Included a reference to the compatible ESP32-S3 firmware in the class summary.
Backward Compatibility:
100% backward compatible. Enable16ChannelMode defaults to false. When disabled, the controller operates exactly as it did before, adhering to the strict 10-channel limit of the standard Wemos D1 Mini Pro. The extended channels and loop iterations are only unlocked if the user explicitly enables them in their cabinet.xml.
```xml
<OutputControllers>
<WemosD1MPStripController>
<Name>ESP32_S3_Controller</Name>
<ComPortName>COM3</ComPortName>
<ComPortTimeOutMs>300</ComPortTimeOutMs>
<ComPortBaudRate>2000000</ComPortBaudRate>
<ComPortOpenWaitMs>300</ComPortOpenWaitMs>
<ComPortHandshakeStartWaitMs>100</ComPortHandshakeStartWaitMs>
<ComPortHandshakeEndWaitMs>100</ComPortHandshakeEndWaitMs>
<SendPerLedstripLength>false</SendPerLedstripLength>
<UseCompression>true</UseCompression>
<Enable16ChannelMode>true</Enable16ChannelMode>
<ComPortDtrEnable>false</ComPortDtrEnable>
<TestOnConnect>false</TestOnConnect>
<NumberOfLedsStrip1>144</NumberOfLedsStrip1>
<NumberOfLedsStrip2>144</NumberOfLedsStrip2>
<NumberOfLedsStrip3>24</NumberOfLedsStrip3>
.......
<NumberOfLedsStrip11>256</NumberOfLedsStrip11>
<NumberOfLedsStrip12>256</NumberOfLedsStrip12>
<NumberOfLedsStrip13>222</NumberOfLedsStrip13>
<NumberOfLedsStrip14>256</NumberOfLedsStrip14>
<NumberOfLedsStrip15>256</NumberOfLedsStrip15>
<NumberOfLedsStrip16>222</NumberOfLedsStrip16>
</WemosD1MPStripController>
</OutputControllers>
```
…iversal High-Availability Watchdog for WemosD1MPStripController This pull request introduces a massive performance and resilience overhaul to the `WemosD1MPStripController` class. It adds support for the ESP32-S3 architecture (up to 16 channels, bulk data streaming) and completely refactors the hardware communication layer to prevent thread crashes caused by transient Windows USB latency or electromagnetic interference (EMI). **The resilience updates apply to ALL controllers using this class (ESP8266, ESP32-S2, ESP32-S3, Teensy).** ### ✨ New Features (S3 / Custom Firmware specific) * **16-Channel ESP32-S3 Support:** Added properties to unlock channels 11 through 16 for high-end setups, while maintaining 100% backward compatibility with standard 10-channel Wemos D1 setups. * **Bulk Stream Mode (`EnableBulkMode`):** Replaces the sequential, per-strip ping-pong handshake with a highly efficient, single-payload stream (`W` command). In classic mode, updating e.g. 8 channels adds ~8ms of pure USB overhead (approx. 1ms per channel just for the ACK roundtrip and waiting for the next strip's data), plus the additional delay of the final 'O' (latch) command. Bulk mode completely eliminates this latency by packing the entire cabinet's data into one payload, drastically increasing FPS for large cabinets. ### 🛡️ Universal High-Availability & Resilience Architecture (Applies to all Hardware) Historically, the plugin handled *any* missed serial ACK (even due to a 2ms Windows CPU spike) by throwing a fatal `TimeoutException`, killing the updater thread, and attempting a full controller re-initialization. This caused visible stuttering or permanent LED freezing. This PR introduces a modern, fault-tolerant approach: * **Streaming Soft-Fail Mitigation:** Missing an ACK (`A`) after a data frame (`W`, `Q`, `O`) no longer crashes the thread. Instead, the timeout is gracefully caught, the COM port buffers are aggressively purged (`DiscardInBuffer`), and the thread immediately proceeds to the next frame. A dropped frame at 90+ FPS is invisible to the human eye; a full thread crash is not. * **Optimized Timeouts:** R/W timeouts have been reduced to `50ms` (down from 300ms). This safely accommodates microcontroller DMA reinitialization windows while ensuring that skipped frames due to Windows background tasks do not cause noticeable physical stutters in the cabinet. * **Hardware Watchdog (EMI & COM-Port Drop Protection):** In a real pinball cabinet, electromagnetic interference (EMI) from contactors or USB hub glitches can cause the virtual COM port to drop for a split second. This triggers a fatal `IOException`. The new Watchdog layer catches this and initiates an intelligent auto-reconnect routine. It will make **5 attempts over a 15-second window** to reclaim the port, resend the `Z` setup configuration, and seamlessly resume the render loop. It is so robust that you can literally unplug the controller during gameplay, plug it back in, and the LEDs will resume instantly. ### 💡 Why this deprecates the "DTR Reset" workaround for ESP32-S2/S3 Previously, users of ESP32-S2 and ESP32-S3 (native USB CDC) were advised to use `DTR=true` to force a hardware/USB stack reset upon reconnection when a frame was dropped. **This is no longer necessary or recommended.** Tearing down a perfectly healthy physical connection and resetting the microcontroller just because Windows delayed an ACK caused massive lag spikes and blackouts. By implementing the Soft-Fail architecture, the connection remains established, the microcontroller stays in its fast-loop, and transient desynchronizations are handled purely via buffer purging.
|
Title: Resilience Update: Universal High-Availability Watchdog for WemosD1MPStripController (last commit) 🛡️ Universal High-Availability & Resilience Architecture (Applies to all Hardware)Historically, the plugin handled any missed serial ACK (even due to a 2ms Windows CPU spike) by throwing a fatal
💡 Why this deprecates the "DTR Reset" workaround for ESP32-S2/S3Previously, users of ESP32-S2 and ESP32-S3 (native USB CDC) were advised to use recommended: |
Vroonsh
left a comment
There was a problem hiding this comment.
You shouldn't totally replace wemosd1 controller but derive from it instead.
That what's been done for wemosd1 based on teensycontroller to keep retro compatibility and modularity solid.
And also it eases the review.
…Input Lag) Resilience Update: Watchdog & EMI Protection for PinOne NamedPipeServer (Fixes massive Input Lag) **Description:** This pull request addresses a critical architectural flaw in the `NamedPipeServer.cs` communication layer that causes severe input lag (often up to 3+ seconds) and stuck contactors when experiencing brief electromagnetic interference (EMI) or transient USB latency. ### 🐛 The Problem Previously, if the `NamedPipeServer` encountered *any* timeout or write error on the COM port (e.g., due to an EMI glitch from a firing contactor dropping the USB port for a millisecond), it handled the exception by completely destroying the server instance (`isRunning = false`). This forced the `PinOneCommunication` client (running on the DOF thread) to sleep for 300ms and attempt to rebuild the entire server. If the port wasn't immediately ready, this loop repeated. Just 10 dropped frames resulted in a cascading **3-second complete freeze** of the DOF pipeline, causing massive flipper button lag and stuck solenoids because the "turn off" commands were caught in the traffic jam. ### 🛡️ The Solution (High-Availability Watchdog) This update refactors the `NamedPipeServer` to gracefully handle hardware drops without tearing down the IPC (Inter-Process Communication) pipe: * **Soft-Fail Mitigation:** R/W timeouts have been reduced from an excessive `500ms` down to `50ms`. If a `TimeoutException` occurs, the server silently drops the frame instead of crashing. * **Continuous Client Acknowledgment:** The most important fix: The server now *always* returns an `"OK"` response to the client, even if a frame was dropped. This entirely prevents the DOF thread from entering its fatal 300ms sleep/reconnect loop. * **EMI Auto-Recovery:** If a hard exception occurs (e.g., COM port dropped due to EMI), the server safely closes the dead serial handle. On the very next incoming `WRITE` request, it automatically re-opens the port seamlessly under the hood. * **Comprehensive Logging:** Added distinct console warnings (`[PinOne Watchdog]`) to help users diagnose their physical hardware (distinguishing between minor USB stutters and full physical EMI drops) without interrupting gameplay. With these changes, the PinOne plugin acts like a tank. You can physically unplug the USB cable mid-game, plug it back in, and it will gracefully resume sending output data immediately once Windows re-enumerates the port, without ever lagging the flipper buttons.
Thanks for the feedback! You make a very valid point regarding modularity and retro compatibility. However, the Watchdog and Soft-Fail (EMI protection) are fundamental stability fixes that solve the COM-Port crashing issue for all existing hardware (Wemos D1, Teensy, etc.), not just the ESP32. Here is my proposed approach to keep the review clean and modular: I will revert the new specific features (16-Channels, Bulk-Mode variables) from the base WemosD1MPStripController.cs. I will only keep the non-breaking try/catch Watchdog and Soft-Fail logic in the base class, so every existing user benefits from the rock-solid stability without changing the structural logic. I will create a new derived class (e.g., Esp32StripController : WemosD1MPStripController). This class will introduce the new properties (Enable16ChannelMode, EnableBulkMode) and override the specific functions needed for bulk streaming. Does this split sound like a solid plan to you? I will update the PR accordingly! |
### 🛡️ Universal High-Availability & Resilience Architecture (Applies to all Hardware) Historically, the plugin handled *any* missed serial ACK (even due to a 2ms Windows CPU spike) by throwing a fatal `TimeoutException`, killing the updater thread, and attempting a full controller re-initialization. This caused visible stuttering or permanent LED freezing. This PR introduces a modern, fault-tolerant approach: * **Streaming Soft-Fail Mitigation:** Missing an ACK (`A`) after a data frame ( `Q`, `O`, `R`) no longer crashes the thread. Instead, the timeout is gracefully caught, the COM port buffers are aggressively purged (`DiscardInBuffer`), and the thread immediately proceeds to the next frame. A dropped frame at high FPS is invisible to the human eye; a full thread crash is not. * **Optimized Timeouts:** R/W timeouts have been reduced to `50ms` (down from 300ms). This safely accommodates microcontroller DMA reinitialization windows while ensuring that skipped frames due to Windows background tasks do not cause noticeable physical stutters in the cabinet. * **Hardware Watchdog (EMI & COM-Port Drop Protection):** In a real pinball cabinet, electromagnetic interference (EMI) from contactors or USB hub glitches can cause the virtual COM port to drop for a split second. This triggers a fatal `IOException`. The new Watchdog layer catches this and initiates an intelligent auto-reconnect routine. It will make **5 attempts over a 15-second window** to reclaim the port, resend the `Z` setup configuration, and seamlessly resume the render loop. It is so robust that you can literally unplug the controller during gameplay, plug it back in, and the LEDs will resume instantly. ### 💡 Why this deprecates the "DTR Reset" workaround for ESP32-S2/S3 Previously, users of ESP32-S2 and ESP32-S3 (native USB CDC) were advised to use `DTR=true` to force a hardware/USB stack reset upon reconnection when a frame was dropped. **This is no longer necessary or recommended.** Tearing down a perfectly healthy physical connection and resetting the microcontroller just because Windows delayed an ACK caused massive lag spikes and blackouts. By implementing the Soft-Fail architecture, the connection remains established, the microcontroller stays in its fast-loop, and transient desynchronizations are handled purely via buffer purging. recommended: `<ComPortTimeOutMs>50</ComPortTimeOutMs>` <!-- Reduced from 300ms for Soft-Fail Architecture -->
Change to AdressableLedStrip
It is designed to stream addressable LED data via UDP to microcontrollers (specifically optimized for ESP32-S3 with W5500 LAN modules or WiFi setups). By eliminating USB bottlenecks and implementing a custom packet-pacing algorithm, this controller easily handles large, asymmetrical LED layouts (e.g., 16 strips with varying lengths up to thousands of LEDs) while maintaining a rock-solid framerate of 100+ FPS without triggering buffer overflows on the receiver end.
**Title:** Add Esp32StripController in AdressableLedStrip namespace **Description:** This pull request introduces adds support for the ESP32-S3 architecture (up to 16 channels, bulk data streaming). ### ✨ New Features (S3 / Custom Firmware specific) * **16-Channel ESP32-S3 Support:** Added properties to unlock channels 11 through 16 for high-end setups. * **Bulk Stream Mode (`EnableBulkMode`):** Replaces the sequential, per-strip ping-pong handshake with a highly efficient, single-payload stream (`W` command). In classic mode, updating e.g. 8 channels adds ~8ms of pure USB overhead (approx. 1ms per channel just for the ACK roundtrip and waiting for the next strip's data), plus the additional delay of the final 'O' (latch) command. Bulk mode completely eliminates this latency by packing the entire cabinet's data into one payload, drastically increasing FPS for large cabinets.
|
Title: Resilience Update: Watchdog & EMI Protection for PinOne NamedPipeServer (Fixes massive Input Lag) Description: This pull request addresses a critical architectural flaw in the 🐛 The ProblemPreviously, if the 🛡️ The Solution (High-Availability Watchdog)This update refactors the
With these changes, the PinOne plugin acts like a tank. You can physically unplug the USB cable mid-game, plug it back in, and it will gracefully resume sending output data immediately once Windows re-enumerates the port, without ever lagging the flipper buttons. |
|
Title: Add Esp32S3StripController in AdressableLedStrip namespace Description: This pull request introduces support for the ESP32-S3 architecture (up to 16 channels, bulk data streaming). ✨ New Features (S3 / Custom Firmware specific)
<OutputControllers>
<Esp32S3StripController>
<Name>ESP32_S3_LEDs</Name>
<ComPortName>COM3</ComPortName> <!-- Change it to your comport name -->
<ComPortTimeOutMs>50</ComPortTimeOutMs> <!-- We’re changing it from 300 to 50ms to have a smooth transition in case of an error. -->
<ComPortBaudRate>2000000</ComPortBaudRate>
<ComPortOpenWaitMs>300</ComPortOpenWaitMs>
<ComPortHandshakeStartWaitMs>100</ComPortHandshakeStartWaitMs>
<ComPortHandshakeEndWaitMs>100</ComPortHandshakeEndWaitMs>
<SendPerLedstripLength>false</SendPerLedstripLength>
<UseCompression>true</UseCompression>
<ComPortDtrEnable>false</ComPortDtrEnable> <!-- Leave it on false. No need for dtr reset anymore -->
<TestOnConnect>false</TestOnConnect>
<!-- New S3 functions -->
<Enable16ChannelMode>true</Enable16ChannelMode>
<EnableBulkMode>true</EnableBulkMode>
<NumberOfLedsStrip1>300</NumberOfLedsStrip1>
<NumberOfLedsStrip2>300</NumberOfLedsStrip2>
<NumberOfLedsStrip3>150</NumberOfLedsStrip3>
<NumberOfLedsStrip4>50</NumberOfLedsStrip4>
<!-- Strips 5 to 16 can be omitted if not used, or set to 0 -->
</Esp32S3StripController>
</OutputControllers> |
Change name and nameclass
New Features (S3 / Custom Firmware specific) 16-Channel ESP32-S3 Support: Added properties to unlock channels 11 through 16 for high-end setups, while maintaining 100% backward compatibility with standard 10-channel Wemos D1 setups. Bulk Stream Mode (EnableBulkMode): Replaces the sequential, per-strip ping-pong handshake with a highly efficient, single-payload stream (W command). In classic mode, updating e.g. 8 channels adds ~8ms of pure USB overhead (approx. 1ms per channel just for the ACK roundtrip and waiting for the next strip's data), plus the additional delay of the final 'O' (latch) command. Bulk mode completely eliminates this latency by packing the entire cabinet's data into one payload, drastically increasing FPS for large cabinets.
|
Hey! Just a quick heads-up regarding the massive diffs in this PR: I updated the modified files by uploading them directly via the web interface, which unfortunately overwrote the line endings (CRLF to LF conversion). Git now incorrectly shows WemosD1MPStripController.cs and NamedPipeServer.cs as completely rewritten. To save you time during the review, here is exactly what actually changed logically: WemosD1MPStripController.cs: NamedPipeServer.cs (PinOne): Sorry for the messy diff on the existing files! I hope this makes reviewing the actual logic easier. |
The Wemos plugin has been restored to its original state with the original logic. But with a watchdog. A bug in my original version caused all data to be sent to the first strip. The functions of the watchdog were commented out. The timeouts were still hard-coded in the code. These were forgotten to be removed.
The original version inherited the bug from my faulty Wemos plugin. A big part of the Wemos logic is now in the ESP32 S3 plugin so that the 16 channels work correctly.
Pull Request Title:
Add FastUdpController: High-Performance Network Streaming for Addressable LEDs
Description:
It is designed to stream addressable LED data via UDP to microcontrollers (specifically optimized for ESP32-S3 with W5500 LAN modules or WiFi setups). By eliminating USB bottlenecks and implementing a custom packet-pacing algorithm, this controller easily handles large, asymmetrical LED layouts (e.g., 16 strips with varying lengths up to thousands of LEDs) while maintaining a rock-solid framerate of 100+ FPS without triggering buffer overflows on the receiver end.
🔄 Protocol Flow & Chunking Logic
Since a full frame of 16 heavily populated LED strips quickly exceeds the standard network MTU (1500 bytes), the controller processes the output as follows:
📦 Packet Structure
To ensure the microcontroller can reconstruct the frame seamlessly, 3 identification bytes are attached to the very front of every single UDP chunk:
Byte 0Byte 10,1,2...).Byte 2The Payload (Starting at Byte 3 of the first chunk):
The very first chunk of a frame always starts with the 33-byte Layout Header:
1 Byte: Number of max strips (Fixed to 16).32 Bytes: The lengths of strips 1 to 16 (represented as 16x High Byte / Low Byte pairs).Remaining Bytes: The sequential RGB data for all configured LEDs.⚙️ Example
cabinet.xmlConfigurationHere is an example of how a user configures this controller in their cabinet setup. They just need to provide the IP of their ESP32, the UDP Port (default 6454), and the physical amount of LEDs per strip.
This addition is completely isolated, does not interfere with existing serial/Teensy implementations, and provides a much-needed robust network alternative for modern VPin builds. My Project: https://github.com/LSatan/ESP32-S3-VPinball-LED-Software