Why can't we target the browser in .Net MAUI using WebAssembly and WebGL?

Question

Integrating WebGL and WebAssembly (WASM) at a low enough level to support a .NET MAUI/XAML runtime is possible but requires a thoughtful design that bridges the WebAssembly execution model with the rendering and UI framework requirements. Here's how this can be approached:

Key Components

WebGL for Rendering:

WebGL provides hardware-accelerated rendering capabilities that can act as the foundation for the visual layer in a runtime.

Use WebGL to render UI elements defined in XAML, effectively turning XAML components into drawable objects.

WASM for Performance-Critical Runtime Logic:

- WebAssembly can be the core execution engine for .NET MAUI, running the managed runtime (e.g., Mono or CoreCLR compiled to WASM).

   - WASM handles application logic, binding updates, and interpreting XAML into renderable objects.

   **Communication between WASM and WebGL**:

      - WebAssembly handles the computational part (e.g., layout calculations, transformations).

         - JavaScript acts as a bridge to interface with the WebGL context, passing rendering instructions and data (e.g., vertex buffers, texture updates).

         **.NET MAUI Runtime**:

            - The .NET MAUI runtime needs to be compiled to WebAssembly. Tools like **Blazor WebAssembly** already provide a blueprint for running .NET in WASM.

               - Leverage .NET libraries for XAML parsing and processing, feeding the parsed visual tree into the WebGL renderer.

Implementation Steps

Compile .NET MAUI to WebAssembly

Use tools like AOT Compilation (Ahead-of-Time) for .NET runtime.
The runtime (Mono or CoreCLR) should be compiled to WASM along with the application logic.

Parse XAML into a Renderable Tree

Use the .NET XAML parser to process the markup and generate a logical UI tree.
This logical tree is translated into a renderable tree (geometry, layout, styles) that can be drawn using WebGL.

Bridge WebGL and WASM

WebGL Renderer: Build a rendering engine that takes inputs from WASM and issues WebGL draw calls.
- This includes vertex buffers, shaders, and textures for visual components like buttons, labels, and containers.
- Data Passing: Use efficient data structures (e.g., TypedArray) for transferring data between WASM and WebGL via JavaScript.

Event Handling

Use JavaScript to capture browser events (e.g., clicks, keypresses, touch gestures) and forward them to WASM.
WASM processes the events and updates the UI state, which is then reflected in WebGL rendering.

.NET MAUI Abstractions

Map .NET MAUI components (e.g., Button, Label) to their visual counterparts in the WebGL renderer.
Implement layout algorithms (e.g., stack layouts, grids) in WASM to compute the geometry.

Performance Optimization

Minimize WASM-to-JavaScript communication overhead by batching WebGL draw calls.
Optimize WebGL shaders and rendering pipelines for dynamic UI updates.

Tools and Libraries

Emscripten: To compile C# runtime dependencies (e.g., Mono) to WebAssembly.
SkiaSharp or Silk.NET: These libraries can provide abstractions for rendering and can be extended to output WebGL-compatible graphics.
Wasm.Bindings: Use wasm-bindgen for interfacing between WebAssembly and JavaScript.

Challenges

Rendering Complexity: Converting a complex XAML visual tree into WebGL primitives requires custom rendering logic.
Resource Management: Efficiently managing textures, buffers, and shaders in WebGL.
Event Binding: Mapping browser-native events to .NET MAUI-style event handlers.
Performance: Achieving smooth rendering and layout recalculations for dynamic UIs.ntegrating WebGL and WebAssembly (WASM) at a low enough level to support a .NET MAUI/XAML runtime is possible but requires a thoughtful design that bridges the WebAssembly execution model with the rendering and UI framework requirements. Here's how this can be approached: Key Components
1. WebGL for Rendering:
  - WebGL provides hardware-accelerated rendering capabilities that can act as the foundation for the visual layer in a runtime.
  - Use WebGL to render UI elements defined in XAML, effectively turning XAML components into drawable objects.
2. WASM for Performance-Critical Runtime Logic:
  - WebAssembly can be the core execution engine for .NET MAUI, running the managed runtime (e.g., Mono or CoreCLR compiled to WASM).
  - WASM handles application logic, binding updates, and interpreting XAML into renderable objects.
3. Communication between WASM and WebGL:
  - WebAssembly handles the computational part (e.g., layout calculations, transformations).
  - JavaScript acts as a bridge to interface with the WebGL context, passing rendering instructions and data (e.g., vertex buffers, texture updates).
4. .NET MAUI Runtime:
  - The .NET MAUI runtime needs to be compiled to WebAssembly. Tools like Blazor WebAssembly already provide a blueprint for running .NET in WASM.
  - Leverage .NET libraries for XAML parsing and processing, feeding the parsed visual tree into the WebGL renderer.
Implementation Steps
1. Compile .NET MAUI to WebAssembly
- Use tools like AOT Compilation (Ahead-of-Time) for .NET runtime.
- The runtime (Mono or CoreCLR) should be compiled to WASM along with the application logic.
1. Parse XAML into a Renderable Tree
- Use the .NET XAML parser to process the markup and generate a logical UI tree.
- This logical tree is translated into a renderable tree (geometry, layout, styles) that can be drawn using WebGL.
1. Bridge WebGL and WASM
- WebGL Renderer: Build a rendering engine that takes inputs from WASM and issues WebGL draw calls.
- This includes vertex buffers, shaders, and textures for visual components like buttons, labels, and containers.
- Data Passing: Use efficient data structures (e.g., TypedArray) for transferring data between WASM and WebGL via JavaScript.
1. Event Handling
- Use JavaScript to capture browser events (e.g., clicks, keypresses, touch gestures) and forward them to WASM.
- WASM processes the events and updates the UI state, which is then reflected in WebGL rendering.
1. .NET MAUI Abstractions
- Map .NET MAUI components (e.g., Button, Label) to their visual counterparts in the WebGL renderer.
- Implement layout algorithms (e.g., stack layouts, grids) in WASM to compute the geometry.
1. Performance Optimization
- Minimize WASM-to-JavaScript communication overhead by batching WebGL draw calls.
- Optimize WebGL shaders and rendering pipelines for dynamic UI updates.
Tools and Libraries
- Emscripten: To compile C# runtime dependencies (e.g., Mono) to WebAssembly.
- SkiaSharp or Silk.NET: These libraries can provide abstractions for rendering and can be extended to output WebGL-compatible graphics.
- Wasm.Bindings: Use wasm-bindgen for interfacing between WebAssembly and JavaScript.
Challenges
1. Rendering Complexity: Converting a complex XAML visual tree into WebGL primitives requires custom rendering logic.
2. Resource Management: Efficiently managing textures, buffers, and shaders in WebGL.
3. Event Binding: Mapping browser-native events to .NET MAUI-style event handlers.
4. Performance: Achieving smooth rendering and layout recalculations for dynamic UIs.

Answer

Blazor already fills this need.

Maui UI is a cross platform wrapper around native controls. It does not have a low level UI element implementations which a WebGL implementation would require. As Maui is supposed to have native feel, you would need to have theming support. also this implementation would not support any native UI controls which are often required.

does not seem like a practical project. A better project is an updated Maui UI that is competitive with SwiftUI and Jetpack which are DSL based rather than XML.

Answer

I guess you have not developed in Silverlight. HTML is DOM based, I am not building hacked documents that look like apps, I'm building applications, therefore I want to use XAML, have you ever developed in XAML before? React = HACK, Angular = HACK, Vue = Hack.

XAML: The Application-First Approach

Purpose-Built for Applications: XAML (eXtensible Application Markup Language) is a declarative UI language explicitly designed for building applications, particularly in the Microsoft ecosystem (e.g., WPF, UWP, and Silverlight). It emphasizes:

Separation of Concerns: Clean separation of UI and logic (via MVVM).
- Rich Tooling: Tight integration with tools like Visual Studio and Blend, which provide a first-class development experience.
  - Stateful UI: Built to handle rich, stateful desktop-style application needs.
    - Strong Typing: Leverages .NET's type system, making it easier to avoid runtime errors.
    Application-Centric Philosophy: XAML was designed with the assumption that you're building applications with complex states, user interactions, and reusable components—not trying to retrofit app-like behavior into a document structure.

Web-Based Frameworks (React, Angular, Vue)

You refer to these as "hacks," which suggests a viewpoint that these frameworks are retrofitting application-like behavior onto an inherently document-oriented system, the DOM. Here's why:

HTML Origins:

HTML was not originally designed for application development; it was created for displaying static documents (hypertext).

CSS and JavaScript were later bolted on to style and add interactivity, leading to significant limitations in structure and state management.

Frameworks to Bridge the Gap:

- React, Angular, and Vue attempt to make the DOM behave like a stateful, interactive UI framework through a combination of virtual DOMs, reactive programming models, and component hierarchies.

   - While powerful, these frameworks are ultimately abstracting the document-oriented foundation of the web.

   **Challenges in Tooling and Complexity**:

      - Debugging and tooling in these frameworks often feel bolted-on rather than native.

         - The need for build tools (Webpack, Vite) and polyfills further highlights the complexity of overcoming the limitations of a document-based system.

React, Angular, Vue as "Hacks"

From your perspective, these frameworks are hacks because:

They work around the limitations of the DOM to approximate the experience of an application-first system.
Their state management and component models attempt to mirror features that XAML and other application-first frameworks offer natively.

Comparative Philosophy

XAML: Starts with the premise, "We are building a fully stateful application."

UI elements are inherently stateful and type-safe.
- Tools are designed specifically for application development.
Web Frameworks: Start with the premise, "We are enhancing documents to act like applications."
```
- The DOM is inherently stateless and dynamic.

   - Frameworks must add layers of abstraction to manage state and mimic an app-like structure.
```

Are Web Frameworks Truly Hacks?

While web frameworks like React and Angular require significant effort to overcome the limitations of the DOM, they:

Have evolved into powerful tools capable of building highly interactive applications.
Provide portability across platforms, including browsers and native apps (e.g., React Native).

Still, your argument is valid if you prioritize:

Native application development experiences.
Frameworks designed explicitly for stateful applications.
Development environments where the tooling feels seamless and application-centric.

Silverlight's Legacy

For developers familiar with Silverlight or WPF, the simplicity and elegance of XAML for building applications are hard to beat. However, the web's ubiquity and the push toward browser-based solutions have forced developers to embrace frameworks that are less "clean" in their underlying philosophy. XAML: The Application-First Approach

Purpose-Built for Applications: XAML (eXtensible Application Markup Language) is a declarative UI language explicitly designed for building applications, particularly in the Microsoft ecosystem (e.g., WPF, UWP, and Silverlight). It emphasizes:

Separation of Concerns: Clean separation of UI and logic (via MVVM).
- Rich Tooling: Tight integration with tools like Visual Studio and Blend, which provide a first-class development experience.
  - Stateful UI: Built to handle rich, stateful desktop-style application needs.
    - Strong Typing: Leverages .NET's type system, making it easier to avoid runtime errors.
    Application-Centric Philosophy: XAML was designed with the assumption that you're building applications with complex states, user interactions, and reusable components—not trying to retrofit app-like behavior into a document structure.

Web-Based Frameworks (React, Angular, Vue)

You refer to these as "hacks," which suggests a viewpoint that these frameworks are retrofitting application-like behavior onto an inherently document-oriented system, the DOM. Here's why:

HTML Origins:

HTML was not originally designed for application development; it was created for displaying static documents (hypertext).

CSS and JavaScript were later bolted on to style and add interactivity, leading to significant limitations in structure and state management.

Frameworks to Bridge the Gap:

- React, Angular, and Vue attempt to make the DOM behave like a stateful, interactive UI framework through a combination of virtual DOMs, reactive programming models, and component hierarchies.

   - While powerful, these frameworks are ultimately abstracting the document-oriented foundation of the web.

   **Challenges in Tooling and Complexity**:

      - Debugging and tooling in these frameworks often feel bolted-on rather than native.

         - The need for build tools (Webpack, Vite) and polyfills further highlights the complexity of overcoming the limitations of a document-based system.

React, Angular, Vue as "Hacks"

From your perspective, these frameworks are hacks because:

They work around the limitations of the DOM to approximate the experience of an application-first system.
Their state management and component models attempt to mirror features that XAML and other application-first frameworks offer natively.

Comparative Philosophy

XAML: Starts with the premise, "We are building a fully stateful application."

UI elements are inherently stateful and type-safe.
- Tools are designed specifically for application development.
Web Frameworks: Start with the premise, "We are enhancing documents to act like applications."
```
- The DOM is inherently stateless and dynamic.

   - Frameworks must add layers of abstraction to manage state and mimic an app-like structure.
```

Are Web Frameworks Truly Hacks?

While web frameworks like React and Angular require significant effort to overcome the limitations of the DOM, they:

Have evolved into powerful tools capable of building highly interactive applications.
Provide portability across platforms, including browsers and native apps (e.g., React Native).

Still, your argument is valid if you prioritize:

Native application development experiences.
Frameworks designed explicitly for stateful applications.
Development environments where the tooling feels seamless and application-centric.

Silverlight's Legacy

For developers familiar with Silverlight or WPF, the simplicity and elegance of XAML for building applications are hard to beat. However, the web's ubiquity and the push toward browser-based solutions have forced developers to embrace frameworks that are less "clean" in their underlying philosophy.

Share via

Why can't we target the browser in .Net MAUI using WebAssembly and WebGL?

2 answers

Your answer