High-quality video conferencing relies on participants being able to see and hear each other clearly, without any disruptions or distortions. The final call quality depends on various factors, including device capabilities and the quality of the network connection between the sender and receiver.

To evaluate the quality of a WebRTC call, it’s crucial to identify key metrics that offer insights into the call’s performance. These metrics reveal what’s happening behind the scenes, allowing us to diagnose and resolve issues effectively. Common problems like choppy audio, video pixelation, and sync issues can be traced by analyzing these metrics. By determining whether the issue originates from the sender’s or receiver’s side, we can take targeted steps to improve the call quality.

In this blog, we’ll explore critical metrics provided by the WebRTC getStats() API that help measure and debug call quality issues. We’ll first focus on the sender side, explaining how to interpret these metrics and offering practical examples to diagnose and improve call quality.

## Key Metrics on the Sender Side

To start with, let's examine the metrics that impact the quality on the publisher's side. These metrics are found in the `RTCOutboundRtpStreamStats` section of WebRTC's `RTCStatsReport`, which is retrieved using the `getStats()` API. For the most accurate data, it's recommended to call the `getStats()` API every second.

### Frame Width and Height

`frameWidth` and `frameHeight` represent the width and height of the last encoded frame respectively. The video resolution, calculated by multiplying width and height, determines the clarity and detail of the video 

- **`frameWidth`:** The width of the video frames in pixels.
- **`frameHeight`:** The height of the video frames in pixels

Higher resolution (larger frame width and height) generally means better video quality but also requires more bandwidth and processing power.

**Purpose of measuring frame width and height**

- **Diagnosing Quality Issues:** If the video quality is poor, checking the `frameWidth` and `frameHeight` can help determine if the issue is related to low resolution.
- **Monitoring Performance:** These metrics can be used to monitor the performance and resource usage of the video stream. For instance, streaming at higher resolutions can increase CPU and network load.
- **Adapting to Network Conditions:** WebRTC can adapt to changing network conditions by adjusting the resolution. Monitoring these metrics can help understand how the application is adapting to bandwidth availability.

### Frame Rate or Frames Per Second (FPS)

`framesPerSecond` represents the number of video frames being processed per second.

**Purpose of measuring `framesPerSecond`**

- **Video Smoothness:** Higher FPS values result in smoother video playback, which is important for user experience.
- **Performance Monitoring:** Monitoring FPS can help in diagnosing performance issues, such as dropped frames or video lag.
- **Adaptation to Conditions:** WebRTC can adjust the frame rate based on network conditions and device capabilities. Monitoring FPS helps understand how the application adapts to these changes.

### Bytes Sent

`bytesSent` metric represents the total number of bytes transmitted over a specific RTP stream, including both the payload (audio or video data) and the RTP packet headers.

**Purpose of measuring `bytesSent`**

1. **Bandwidth Monitoring:** Tracking the bytesSent metric over time provides insights into the bandwidth consumption of the outgoing stream, which is essential for assessing network load and performance.
2. **Bitrate Calculation:** You can calculate the bitrate (data rate) by measuring the change in `bytesSent` over time. 
3. **Troubleshooting:** Monitoring `bytesSent` can help in diagnosing issues related to packet loss, network congestion, or other transmission problems.

### Outgoing Bitrate

The **upload bitrate** or **outgoing bitrate** represents the rate at which data is transmitted from the local peer to the remote peer, typically measured in bits per second (bps). A stable bitrate is key to ensuring smooth audio/video playback and a consistent user experience.

**Purpose of measuring outgoing bitrate**

1. **Media Quality:** Outgoing bitrate directly affects media quality. Higher bitrates generally lead to better video resolution and clarity, as well as improved audio fidelity, ensuring a superior viewing and listening experience.
2. **Bandwidth Utilization:** Monitoring outgoing bitrate provides insights into the application’s bandwidth usage, helping maintain operations within the network’s capacity and preventing congestion.
3. **Identifying Performance Issues:** Persistently low outgoing bitrates may indicate network issues, server limitations, or client-side bottlenecks, making it a valuable metric for diagnosing and resolving performance problems.

By tracking outgoing bitrate, developers can optimize network and computational resources, adjusting video resolution, frame rate, or codec settings based on the available bandwidth and current network conditions.

### Packets Lost

`packetsLost` metric represents the cumulative count of RTP (Real-time Transport Protocol) packets that were expected to arrive at the receiver but did not. This count starts from the beginning of the RTP stream and includes all packets lost during the session. 

**Causes of Packet Loss**

1. **Network Congestion:** High traffic can cause packets to be dropped if network devices cannot handle the volume.
2. **Network Errors:** Physical issues like damaged cables or faulty hardware can lead to packet loss.
3. **Interference:** Wireless networks are particularly susceptible to interference from other devices and obstacles.
4. **Route Changes:** Changes in the network routing can result in packets being lost in transition.

**Effects of Packet Loss**

1. **Audio Issues:** Missing packets in audio streams can cause glitches, dropouts, or periods of silence.
2. **Video Issues:** Missing packets in video streams can result in pixelation, freezing, or frame drops.
3. **Overall Quality:** High packet loss can degrade the overall user experience, making communication difficult and unpleasant.

### Round Trip Time (RTT)

Round Trip Time (RTT) / `roundTripTime` is the total time for a data packet to travel from the sender to the receiver and back. It is a key metric for measuring latency in a WebRTC connection.

Higher RTT increases latency, causing noticeable delays in conversations, while lower RTT reduces latency, enabling more natural, real-time interactions.

**Typical RTT values and associated experience:**

| Experience | RTT |
| --- | --- |
| Excellent | < 150ms |
| Acceptable | 150ms - 300ms |
| Problematic | > 300ms |

### Jitter

Jitter refers to the variation in time delay between RTP packets arriving. In an ideal network, packets would arrive at evenly spaced intervals. However, in real-world networks, various factors cause packets to experience different delays, leading to jitter.

**Causes of Jitter**

1. **Network Congestion:** High traffic can cause delays as packets wait in queues before being transmitted.
2. **Route Changes:** Different packets may take different paths to reach the destination, resulting in variable delays.
3. **Packet Loss and Retransmission:** Lost packets that need to be retransmitted add to the delay variability.
4. **Interference:** Wireless networks, in particular, can experience interference that affects packet delivery times.

**Effects of Jitter**

1. **Poor Audio/Video Quality:** Jitter can cause audio to sound choppy or video to appear jerky.
2. **Latency:** Increased jitter often leads to higher overall latency, impacting real-time communication quality.
3. **Buffering:** To compensate for jitter, buffers are used, but excessive jitter can overwhelm buffers and cause delays.

For most real-time communication applications, jitter values can follow this :

| Jitter | Range | Inference |
| --- | --- | --- |
| Optimal | < 30ms | Values below 30 ms are generally considered excellent and indicate a stable network with minimal variation in packet delay. |
| Acceptable | 30ms - 50ms | While some variation in packet arrival times is present, it is often manageable with jitter buffers and other compensatory mechanisms. |
| Borderline | 50ms - 100ms | Values in this range might start causing noticeable issues, such as minor audio dropouts or video frame skips. Adjustments may be needed to maintain quality. |
| Problematic | > 100ms | Generally a sign of serious network issues, values above 100 ms can lead to significant degradation in communication quality, including choppy audio, noticeable delays, and poor video synchronization.  |

### Total Packet Sent delay

`totalPacketSendDelay` metric represents the total time (in seconds) that packets have spent in the send buffer, waiting to be sent. It essentially measures the delay introduced at the sender side before the packets are transmitted over the network. This is present in **outbound-rtp (kind = video/audio).**

**Purpose of measuring `totalPacketSendDelay`**

1. **Assessing Network Congestion:**
    - High `totalPacketSendDelay` values can indicate network congestion or bottlenecks at the sender's end, where packets are queued up waiting to be sent.
2. **Diagnosing Latency Issues:**
    - Monitoring `totalPacketSendDelay` helps diagnose and troubleshoot latency issues. If packets are delayed before even being sent, it contributes to the overall end-to-end latency experienced by the user.
3. **Optimizing Application Performance:**
    - Understanding the delays in packet transmission allows developers to optimize the application. This might involve adjusting the bitrate, modifying the encoding settings, or enhancing the network infrastructure to reduce send delays.
4. **Encoding and Processing Delays:** 
    - The time taken to encode and process media before transmission can add to the delay.
5. **Hardware Performance:** 
    - The performance of the sender's hardware can impact how quickly packets are prepared and sent.

In an ideal scenario, `totalPacketSendDelay` should be as low as possible.

### Available Outgoing Bitrate

`availableOutgoingBitrate` represents the estimated maximum bitrate that can be sent on the current network path and provides insights into the network's capacity to handle outgoing media streams. This is calculated using the underlying congestion control (gcc) algorithm. Please not that this does not indicate the exact upload bandwidth of the network connection but an approximation based on the amount of bitrate it is uploading. 

**Purpose of measuring `availableOutgoingBitrate`**

1. **Network Capacity Monitoring:**
    - `availableOutgoingBitrate` helps in understanding the current network capacity for outgoing media streams. It indicates how much data can be sent per second without causing congestion or packet loss.
2. **Adaptive Bitrate Management:**
    - WebRTC applications often use adaptive bitrate algorithms to adjust the quality of audio and video streams based on network conditions. Monitoring `availableOutgoingBitrate` allows these algorithms to dynamically adjust the bitrate to match the available network capacity, ensuring smooth and high-quality media transmission.
    - **Buffer:** Maintaining a buffer above the required bitrate helps accommodate network fluctuations.
3. **Troubleshooting and Diagnostics:**
    - Sudden changes in `availableOutgoingBitrate` can signal network issues such as bandwidth fluctuations, congestion, or other problems. This information is valuable for diagnosing and troubleshooting network-related issues in real-time communication applications.

If this value is lesser than the upload bitrate that is configured, then there would be publish side degradations observed , i.e , reduction in bitrate and/or fps being published 

### Quality Limited Durations

`qualityLimitationDurations` is a metric that provides detailed information about the durations during which the quality of a media stream was limited due to various reasons. This metric is part of the `RTCOutboundRtpStreamStats` dictionary and is particularly useful for diagnosing and understanding the factors affecting the quality of a media stream. This is present in **candidate-pair(state = succeeded).**

**Components of `qualityLimitationDurations`**

The `qualityLimitationDurations` metric includes multiple sub-metrics that indicate the time spent under different types of quality limitations:

1. **`bandwidth`:** Duration during which the quality was limited due to insufficient bandwidth.
2. **`cpu`:** Duration during which the quality was limited due to high CPU usage.
3. **`other`:** Duration during which the quality was limited due to other reasons that don't fall under bandwidth or CPU constraints.

**Purpose of measuring `qualityLimitationDurations`**

1. **Diagnosing Quality Issues:**
    - By breaking down the durations of quality limitations into specific categories (bandwidth, CPU, other), developers can pinpoint the exact cause of quality degradation.
2. **Performance Monitoring:**
    - This metric helps in monitoring the performance of the WebRTC application over time, providing insights into how often and for how long the media quality is being compromised.
3. **Optimization and Tuning:**
    - Understanding the root causes of quality limitations allows developers to optimize and tune the WebRTC application. For example, if bandwidth is often the limiting factor, adaptive bitrate techniques can be used to manage the media quality better.

To effectively troubleshoot publisher-side issues in a WebRTC call, it’s crucial to monitor the above key metrics. One approach is to capture metrics from the getStats() API at regular intervals (e.g., every second), then aggregate or average the data over a longer period (e.g., 30 seconds) before uploading it to the server for analysis. Now that we’ve some idea about the publisher side metrics, let’s see how we can use them to debug actual webRTC calls.

## Case Study

Executing a seamless video conferencing is not as straightforward as it might seem. Some issues are frequent and there is a set protocol to debugging those, but we also run into interesting problems and edge cases often. At 100ms, monitoring and addressing these metrics has become a routine part of our operations. To facilitate this process, most of the metrics mentioned earlier are directly available to customers through the 100ms Dashboard. We’ve built a **powerful call quality dashboard** where these statistics are plotted as time series graphs, making it easier for users to diagnose and resolve issues quickly. This data is presented at both the session and participant levels, allowing for granular insights into call performance.

We’ll take a small case study to understand how we usually debug real life call quality and experiential issues. The following snapshots are from the 100ms Peer Insights Dashboard, showing data for a participant who encountered degraded publishing quality.

### Problem Statement and Impact

There was a problem where the video being seen by other participants from the participant A’s end wasn’t of good quality, which made it hard for them to view the participant A clearly.

### Observations

We begin by examining the publishing bitrate graph, which indicates a significant drop in the bitrate being published around 06:18 HRS. Simultaneously, the FPS (frames per second) values also show a noticeable decline during this period.

This drop suggests that the video quality being transmitted was compromised at that time. As a result, the receivers / viewers likely experienced pixelated video due to the low bitrate and encountered choppy playback because of the reduced FPS.

This aligns with the observations shared by others participants. We’ll have to ask couple of what and why questions to understand the root cause.

![observation graphs.png](https://storage.googleapis.com/100ms-cms-prod/cms/obs_graphs_a25b9e61f2/obs_graphs_a25b9e61f2.png?updated_at=2024-09-06T06:55:58.527Z)

### What caused the sudden drop in bitrate and FPS?

To diagnose the issue further, we examine the network metrics. Key observations are as follows:

- The upload network experienced packet drops starting around 06:18 HRS, coinciding with the time of degraded video quality.
- There was a sudden spike in RTT (Round-Trip Time) values at the same timestamp, reaching as high as 6000ms.
- Packet loss occurred in bursts, with levels reaching up to 100%, as indicated in the graph.
- Although RTT values eventually returned to normal, persistent packet loss continued to affect the network.

![cause graphs.png](https://storage.googleapis.com/100ms-cms-prod/cms/cause_graphs_f7bf8fd000/cause_graphs_f7bf8fd000.png?updated_at=2024-09-06T06:55:58.529Z)

As a result, the video publisher had to reduce both bitrate and FPS to adapt to the sudden network degradation, leading to a compromised publishing as well as viewer experience. This tells us that the culprit in question is the network, but this begs another question:

### What led to the sudden network degradation, and what specific issues occurred with the network?

The network degradation appears to have been caused by congestion in the upload network. This is indicated by the jitter metric, which shows significant deviation from its normal values around the same timestamp (06:18 HRS). The increased jitter suggests that packets were arriving at the receiver end in an unevenly spaced manner, leading to packet loss.

![jitter graphs.png](https://storage.googleapis.com/100ms-cms-prod/cms/jitter_graphs_b5b87dec57/jitter_graphs_b5b87dec57.png?updated_at=2024-09-06T06:55:58.261Z)

These network issues, evidenced by the sudden spikes in jitter, RTT, and packet loss, likely caused the upload quality to deteriorate, resulting in the observed drops in bitrate and FPS. This combination of factors indicates that the peer was experiencing substantial network instability, which directly impacted the quality of the video being published.

### How can we address the issue of degraded video quality caused by network instability?

While sudden and significant network drop-offs are often beyond our control, we can optimize the overall experience on consistently poor networks by adjusting parameters such as available bitrate, resolution, and FPS. These adjustments help maintain consistent, if not great video quality under challenging network conditions.

## Conclusion

While we’ve focused on sender-side metrics in this blog, it’s important to remember that the receiving side plays an equally crucial role in determining the final call quality.

In our next blog, we will dive into the key metrics on the receiving / subscriber side. We will explore how these stats impact call quality and work towards developing a formula to qualitatively measure video quality at the receiver end. Understanding both sides will enable us to provide a more comprehensive approach to optimizing video call experiences.

To explore and experience these metrics firsthand, log in to the 100ms dashboard and conduct a demo call to generate real-time data. You can learn more about Peer Insights by visiting [this link](https://www.100ms.live/docs/get-started/v2/get-started/insights/peerinsights).

thumbnail_Measuring_Call_Quality_-_1.png

Measuring_Call_Quality_-_1.png

Measuring WebRTC Call Quality - Part 1

*A video call is only as strong as its weakest link. For smooth, engaging communication to occur, each participant must enjoy a flawless experience. Participants bring diverse devices, networks, locations into the mix - introducing variability in conditions. However, with diverse devices, networks, and locations involved, its introduces variability in conditions. A mobile user might endure constant buffering while a desktop stream remains smooth. Or one peer's video turns pixelated while others maintain crisp HD quality.*

*Imagine trying to troubleshoot such an issue. When video call quality degrades, pinpointing the cause can be a daunting task due to the complexity of the different environments involved. We've all been there – sifting through dense, time-intensive video logs, trying to make sense of the jumble of data.*

*But what if there were a way to cut through the noise and gain meaningful insights into the root causes of such issues? A way to transform those complex logs into a readable, user-friendly format that highlights the progression of each peer's call experience?*

## Introducing Call Insights
At 100ms, we believe in abstracting complexities. Following this principle, we wanted to help developers debug video call sessions and turn complicated logs into meaningful information. We're excited to introduce Peer and Session Insights on the 100ms dashboard. This tool provides a quick way to understand and troubleshoot a session and each participant's experience.
Complex logs are converted into easy-to-read events with timestamps, showing what happened during each participant's call. Now, both developers and PMs can diagnose issues, identify problems, and strategically improve call quality based on information reported by participants.
In this article, we will go through key parts of this tool that can be used to troubleshoot or simply review how a session went for its users. We'll also look into debugging a specific participant's session experience. So, read along!

### How did the session go?
Delivering an exceptional video call experience is key. The 100ms dashboard provides powerful Session Insights tools to review session details in an easy format. Simply go to Sessions, select a session ID, and access comprehensive summaries of what happened during the call. Let's briefly review these:

1. **Session Timeline -** The Session Timeline provides a chronological view of events across all peers in a session.  You can find a specific session by entering the session ID in the 100ms dashboard > Sessions section. The timeline includes:
    1. **Peer Events -** Key events that occurred across peers during the session, such as join, leave or hand raises.
    2. **Recording Events (if enabled) -** Events when recording started, stopped, and ended for the particular session.
    3. **Livestreaming Events (if enabled) -** Events when livestreaming started and stopped. 
    4. **Error Events -** Any session-related errors, such as recording failures or livestream failures, are captured here.
2. **Session Health -** This section provides an overall Connection Quality Score or CQS for all peers who joined the particular session. The score is calculated across all peers based on metrics like packet loss, jitter, and bandwidth, quantifying the impact of network conditions. 
3. **Issue Affected Peers and their Breakdown -** This section summarizes the total number of peers affected of all peers in that session. However, knowing that it was a problematic session is not good enough. number of peers affected by various issues, followed by a breakdown of the peers aggregated by issue type. This information helps identify and troubleshoot any problems that may have occurred during the session. This section also summarizes how many participants experienced each type of issue. This breakdown makes it easier to see which issues were most common and analyze the affected participants.

### Going a level deeper with Peer Insights

1. **Peer Details -** Get context on each peer’s environment with details like their name, user ID, session information, browser, device, and SDK version.
2. **Event Log -** Access a chronological record of all events experienced by the peer, including audio/video muting, device switching, and error occurrences. This visibility into their journey helps pinpoint potential issues.
3. **Experience Issues -** Identify and troubleshoot session-related problems like join failures, disconnections, publishing errors, and subscription failures.
4. **Connection Quality Score -** Gain insights into the peer's network performance trend throughout the call. The score is updated at 15-second intervals using the latest data points captured, offering a comprehensive view of the session's health.
    
    ![Screenshot 2024-01-28 at 2.30.50 PM.png](https://prod-files-secure.s3.us-west-2.amazonaws.com/cc7eac53-8d9e-4a5e-8f11-47296e6762a8/c8857d7b-acf9-4404-a0a4-58ef3c97d18a/Screenshot_2024-01-28_at_2.30.50_PM.png)
    
5. **Network Performance**: Track underlying network conditions that affect media quality, such as bitrate, frame rate, packet loss, round-trip time, jitter, and CPU/network limitations. Understand how these factors influence the peer's experience. Read how Publisher(outgoing) and Subscriber(incoming) media quality is impacted by network factors.
6. **Quality Summaries**: Go beyond the connection quality score with detailed summaries of the peer's publisher and subscriber network quality, including the duration of any issues faced.

### Let's debug a peer

Let’s take an example of a session where a peer faced audio/video issues during a session and debug it on 100ms Dashboard. The peer reported audio choppiness and not being able to view others video.

Participants a.k.a peers in a 100ms session are identified using a unique `peer_id` for a unique session. On 100ms dashboard, you can view all peers who participated in a given session. You can also filter peers using your user ID that you pass to 100ms.

Follow these steps to view Peer details and insights

1. Sign-up or Log in to 100ms Dashboard.  
    
    For existing sessions, go to Sessions; specify or filter sessions using a session ID
    
2. Select a peer using Peer Id or Peer name
3. Click on Peer ID and move to peer details page



### Steps to debug issues faced by a peer

1. Peer Information - 
2. Event Log -
3. **Experience Issues -** No Issues reported indicates that this peer encountered no significant experience issues.
4. **Connection Quality Score -** Average CQS: 3.73 - An average score of 3.73 suggests a fair to good connection quality indicating certain blips during the session. the graphs shows that the peer started facing issues from 20:39 to 20:44:21 impacting connection quality for about for 4 mins 30 seconds in total. Quality Issues: 2 Issues
    1. Publisher Network Issues: 4 minutes 30 seconds in total
        1. Bitrate
        2. FPS is impacted
        3. RTT
        4. Jitter
        5. Quality was limited - reason for publisher network issues
    2. Subscriber Network Issues: 4 minutes 30 seconds in total
        1. incoming bitrate - expected bitrate was more than actual bitrate received
        2. incoming packet loss - lost incoming video from remote peers

## Next On Call Insights

We’ve only just begun unlocking video call insights. While peer-level insights simplifies debugging the lowest level  peer and session level diagnostics, there’s a lot more coming soon to **Call Insights:**

- **Aggregated Insights Across All Sessions**
    - Pinpoint high-quality sessions versus those with quality challenges.
    - Track key usage metrics and trends across all your video sessions.
    - Identify macro-level patterns in devices, browsers, and networks causing widespread issues among peers.

Start debugging sessions and participants today and unlock a new level of visibility into your video call infrastructure. Read more on Peer Insights docs. Stay tuned as the Call Insights journey continues! 👋🏻

thumbnail_Peer_Insights.png

Peer_Insights.png

Introducing Call Insights

Aditya Thakur

In the first part of our series on Chat Moderation in 100ms, we shared some simple techniques for keeping chat in check. You might want to look [here](https://www.100ms.live/blog/chat-moderation-part-1) if you have not seen it.

Now, let's dive into more ways to handle text chats in video apps using 100ms. This time, we'll use cloud services and LLMs to do the job automatically. Just like before, we'll tweak the chat feature of 100ms Prebuilt to show you how it's done.

> Keep in mind, that the methods we're talking about are just one way to help with moderation. They're here to give developers ideas, though there's room for improvement.

## **Getting started**

We'll begin by setting up 100ms Prebuilt for this tutorial as it comes with Chat configured.

- Fork your copy of the [web-sdks](https://github.com/100mslive/web-sdks) repository from [here](https://github.com/100mslive/web-sdks).
- Clone it to your system and open it in a code editor of your choice.
- Run the command `yarn` to validate and resolve packages using the [Yarn package manager](https://yarnpkg.com/).
- Navigate to `web-sdks/packages/roomkit-react/src/Prebuilt/components/Chat/ChatFooter.tsx`

In the 100ms Prebuilt, the message is typed in the Chat Footer and before it is sent it is stored in the `message` variable as follows:

```
const message = inputRef?.current?.value;
```

- Run the `yarn build` command to build the packages.
- Next, navigate to the `prebuilt-react-integration` under the `examples` directory and run the `yarn dev` commands in the terminal, to run the 100ms Prebuilt example locally.

## Using a Cloud Service

Various cloud services offer options for moderating text. For instance, Google Cloud provides [Natural Language API](https://cloud.google.com/natural-language/docs/reference/rest), while Microsoft Azure offers [Content Moderator for text moderation](https://learn.microsoft.com/en-us/azure/ai-services/content-moderator/text-moderation-api), among others. 

By leveraging these cloud-based moderation tools, developers can implement robust chat moderation systems without having to build everything from scratch. This saves time and resources while ensuring that chat conversations remain constructive and free from offensive content thus creating a safer environment for users and also reducing the burden on human moderators.

### Using Google Cloud API

The first step is to find the API and enable it.

- Search for the Cloud Natural Language API in the Google Cloud Console.
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Search_Cloud_Natural_Language_API_435c67a16b/Search_Cloud_Natural_Language_API_435c67a16b.png) 

- Click on the 'Cloud Natural Language API' under Marketplace and click on the 'Enable' button.
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Enabled_Cloud_Natural_Language_API_1d37df179c/Enabled_Cloud_Natural_Language_API_1d37df179c.png) 


- Then, we'll need to create credentials to access the enabled APIs. We create an API key as follows:

<video loop="true" autoplay="autoPlay" id="vid" muted>
<source src="https://storage.googleapis.com/100ms-cms-qa/cms/API_Key_Google_Cloud_Natural_Language_API_50820633a7/API_Key_Google_Cloud_Natural_Language_API_50820633a7.mp4?updated_at=2024-04-23T06:38:13.918Z" type="video/mp4" />
</video>

<br>

To utilize the [Natural Language API](https://cloud.google.com/natural-language/docs/reference/rest), we'll need to send a POST request to the provided endpoint with the API Key as a parameter.

```bash
POST https://language.googleapis.com/v2/documents:moderateText?key=API_KEY
```

The API provides us with a list of label names along with their corresponding confidence values. These labels represent different categories such as "Toxic", "Insult", etc. each with a confidence score indicating the likelihood that the text falls into that category.

For instance, if the confidence score for the "Toxic" category is significantly high, we can alert the user about the potentially harmful content and take measures to mitigate its impact.

```jsx
{
  "moderationCategories": [
    {
      "name": "Toxic",
      "confidence": 0.8611886
    },
    {
      "name": "Insult",
      "confidence": 0.5483594
    },
  ...
  ],
  "languageCode": "en",
  "languageSupported": true
}
```

Additionally, the provided language code indicates the language of the text being analyzed, and the "languageSupported" field confirms whether the API supports the specified language for moderation purposes. This information helps ensure that the moderation process is accurate and effective across different languages.

In our code, we find the message variable. We can pass this message to the API request body, before it is sent on the public chat, as follows:

```jsx
    await fetch(
      'https://language.googleapis.com/v2/documents:moderateText?key=API_KEY',
      {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          document: {
            content: message,
            type: 'PLAIN_TEXT',
          },
        }),
      },
    )
      .then(response => {
        if (!response.ok) {
          ToastManager.addToast({
            title: 'Response not ok. Please try again.',
          });
        }
        return response.json();
      })
      .then(data => {
        const moderationCategories = data.moderationCategories;
        let maxConfidence = 0;
        let maxCategory = '';

        moderationCategories.forEach((category: { confidence: number; name: string }) => {
          if (category.confidence > maxConfidence && category.confidence > 0.8) {
            maxConfidence = category.confidence;
            maxCategory = category.name;
          }
        });

        if (maxCategory !== '') {
          ToastManager.addToast({
            title: 'Message contains ' + maxCategory + ' information. Please refrain from it!',
          });
          return;
        } 
      })
      .catch(error => {
        ToastManager.addToast({
          title: 'There was an error moderating the message. Please try again.' + error,
        });
      });
```

Notice how we iterate through the list of moderation categories, each containing a confidence value and a corresponding name. Once we determine the category with the maximum confidence, we display a message to the user via a toast notification. By providing real-time feedback about potentially problematic content, we contribute to fostering a safer and more respectful online environment.

<video loop="true" autoplay="autoPlay" id="vid" muted>
<source src="https://storage.googleapis.com/100ms-cms-qa/cms/demo_google_cloud_011b34a2c1/demo_google_cloud_011b34a2c1.mp4?updated_at=2024-04-23T06:42:35.571Z" type="video/mp4" />
</video>

### Using Microsoft Azure API

Similar to the Natural Language API, we can utilize Content Moderator's text moderation models to examine text content and manage chat in 100ms.

It also offers various categories with scores, presented in the following format:

```bash
"Classification": {
    "ReviewRecommended": true,
    "Category1": {
        "Score": 1.5113095059859916E-06
    },
    "Category2": {
        "Score": 0.12747249007225037
    }
}
```

Furthermore, in addition to its primary functions, the text moderation system also identifies potential instances of personal data, including common forms such as email addresses, US mailing addresses, IP addresses, and US phone numbers.

Moreover, the text moderation response can provide the text with basic auto-correction applied, if preferred. This feature can help improve the readability and clarity of the text while maintaining its original meaning.

The system can be customized to screen for terms that are tailored to your business requirements. 

For instance, you may have specific terms or phrases that you want to filter out from user-generated content, such as competitive brand names or sensitive information. This customization ensures that the moderation process aligns with your business objectives and helps maintain a consistent brand image across your platform.

> Learn more about Microsoft Azure’s Natural Language API [here](https://learn.microsoft.com/en-us/azure/ai-services/content-moderator/text-moderation-api).
> 

## Using an LLM

Using Large Language Models (LLMs) for chat moderation can sometimes be excessive, primarily due to the associated costs. However, one intriguing use case is to prompt the AI model to adjust the text to generate a message that is safe to share. This approach not only helps mitigate the risks associated with inappropriate or harmful content but also provides a solution for maintaining a positive user experience. 

By automatically adjusting the text to align with community guidelines or platform policies, this method contributes to fostering a safer and more inclusive online environment.

### A local LLM like Gemma

[Gemma](https://ai.google.dev/gemma) is a set of lightweight, advanced open models constructed using the same research and technology behind the Gemini models. The model weights are accessible in two sizes: [Gemma 2B and Gemma 7B](https://www.kaggle.com/models/google/gemma), each accompanied by pre-trained and instruction-tuned variations.

- The first step is to download a supported model and store it within the project directory:

```
<dev-project-root>/assets/gemma-2b-it-gpu-int4.bin

```

- A task can then be created as follows:

```bash
const genai = await FilesetResolver.forGenAiTasks(
    // path/to/wasm/root
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@latest/wasm"
);
const llmInference = await LlmInference.createFromOptions(genai, {
    baseOptions: {
        modelAssetPath: '/assets/gemma-2b-it-gpu-int4.bin'
    },
});
```

Here we use **`createFromOptions()`** function and specify the path of the model with the `baseOptions` object `modelAssetPath` parameter. 

- Finally, run the task as follows:

```
const response = await llmInference.generateResponse(message);
```

The LLM Inference API employs the **`generateResponse()`** function to initiate inferences and provides a string as its output, containing the generated response text.

### An API like Open AI’s API

If the asset file is too large to include with your project, you might opt to utilize an online AI model such as OpenAI's GPT-4 for moderation purposes instead.

Implementing the Open AI’s API for this task is relatively simple - send your message and prompt to the API endpoint, and it will return the edited message if the original was found to be inaccurate or insufficient.

```bash
	 await fetch('https://api.openai.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: 'Bearer OPENAI_API_KEY',
      },
      body: JSON.stringify({
        model: 'gpt-3.5-turbo',
        messages: [
          {
            role: 'user',
            content:
              'You are a chat moderator for a live audio video session. Your task is to make sure the text shared in chat is appropriate. If the user message shared is not appropriate rewrite it to make it follow community guidlines. If it is appropriate, send it back as is. Do not reply with anything else. The user intends to send this message: ' +
              message,
          },
        ],
        temperature: 0.7,
      }),
    })
      .then(response => {
        if (!response.ok) {
          throw new Error('Network response was not ok');
        }
        return response.json();
      })
      .then(data => {
        const completion = data.choices[0].message.content.trim();
        console.log(completion);
        message = completion;
        ToastManager.addToast({
          title: 'Message modified to comply with community guidelines.',
        });
      })
      .catch(error => {
        console.error('There was a problem with the fetch operation:', error);
  });

```

<video loop="true" autoplay="autoPlay" id="vid" muted>
<source src="https://storage.googleapis.com/100ms-cms-qa/cms/demo_of_gpt_api_9231cdccf7/demo_of_gpt_api_9231cdccf7.mp4?updated_at=2024-04-23T09:39:45.217Z" type="video/mp4" />
</video>

<br>

We've explored various options for chat moderation, both through cloud services and utilizing LLMs. Thank you for taking the time to read through it. I trust it’ll provide insights into constructing your chat moderation system.

Should you have any inquiries, our team is ready to assist you on Discord. You can find us [here](https://discord.com/invite/kGdmszyzq2).

thumbnail_Chat Mod - 2.png

Chat Mod - 2.png

How to moderate chat in 100ms? - Part 2

When we create live audio-video experiences online, keeping the environment safe for everyone involved is important. Sometimes, personal info like email or address might accidentally get shared in the chat, or someone might use inappropriate language. While moderators are helpful, it's also good to have automatic moderation for larger groups or one-on-one calls to keep everyone safe.

In this article, we'll explore different ways to automatically manage text chats in video apps using 100ms. We'll start by adjusting the chat feature of 100ms Prebuilt to demonstrate an example.

Remember, the methods we discuss are just one way to help with moderation. They're meant to give developers ideas, but they're not perfect. 

## Getting started

We'll begin by setting up 100ms Prebuilt for this tutorial as it comes with Chat configured.

- Fork your copy of the [web-sdks](https://github.com/100mslive/web-sdks) repository from [here](https://github.com/100mslive/web-sdks).
- Clone it to your system and open it in a code editor of your choice.
- Run the command `yarn` to validate and resolve packages using the [Yarn package manager](https://yarnpkg.com/).
- Navigate to `web-sdks/packages/roomkit-react/src/Prebuilt/components/Chat/ChatFooter.tsx`

In the 100ms Prebuilt, the message is typed in the Chat Footer and before it is sent it is stored in the `message` variable as follows:

```jsx
const message = inputRef?.current?.value;
```

- Run the `yarn build` command to build the packages.
- Next, navigate to the `prebuilt-react-integration` under the `examples` directory and run the `yarn dev` commands in the terminal, to run 100ms Prebuilt example locally.

### Using Regular Expression

The first approach, we look at, is to use a regular expression to omit any personal information being shared in public chat. Before sending the message, we can run it through regular expression to check for personal information like emails, etc.

A simple regex like the one below can be added to the `ChatFooter` component:

```jsx
const emailRegex = /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/;
const containsEmail = emailRegex.test(message);

if (containsEmail) {
  ToastManager.addToast({
    title: 'Message contains personal information. Please do not share personal information!',
  });
  message = message.replace(/([A-Za-z0-9._%+-]+)@/, 'xxxxx@');
}
```

The above would check if the message contains email and show a toast notifying the user that personal information should not be shared. As developers, we can completely omit the message or still allow it to be sent replacing the characters. In the above, code I have chosen to replace the characters before @ in the email with `xxxxx` for demonstration.

Run the `prebuilt-react-integration` in the `examples` directory to view the changes. When trying to send a message with an email, the user would see the following:

<video loop="true" autoplay="autoPlay" id="vid" muted>
<source src="https://storage.googleapis.com/100ms-cms-qa/cms/email_moderation_demo_a4a0281c63/email_moderation_demo_a4a0281c63.mp4?updated_at=2024-04-09T10:11:20.714Z" type="video/mp4" />
</video>

### Using an external package

We can do something similar for phone numbers using regular expressions. However, it can require extensive effort to write code to tell if the number shared by the user is a phone number or just another number. To allow additional checks, instead of writing the code on our own, we can use an external package like the [phone](https://www.npmjs.com/package/phone) ****package ****to check for phone numbers**.**

In the `package.json` of the `roomkit-react` package, add the following under dependencies:

```jsx
"phone": "3.1.42" //Please use the latest version
```

Run the `yarn` command again to validate and resolve the added packages. Return to the `ChatFooter.tsx` file to import the package:

```jsx
import { phone } from 'phone';
```

And, add the following to check for phone numbers and replace them accordingly:

```jsx
const { isValid } = phone(message);

if (isValid) {
  ToastManager.addToast({
    title: 'Message contains personal information. Please do not share personal information.',
  });
  message = message.replace(message, 'xxxxxxxxxx');
}

```

 ![Phone Number moderation in 100ms](https://storage.googleapis.com/100ms-cms-qa/cms/phone_moderation_2c4240d5e8/phone_moderation_2c4240d5e8.png) 

### Using a pre-trained model

TensorFlow.js offers [pre-trained models for common ML tasks](https://www.tensorflow.org/js/models) to add to web and browser-based applications. One such model for text, that enables NLP in a web app, is ‘[Text toxicity detection](https://github.com/tensorflow/tfjs-models/tree/master/toxicity)’.

The toxicity model detects whether the text contains toxic content such as threatening language, insults, obscenities, identity-based hate, or sexually explicit language. 

**Installation**

The following dependency is required in the `package.json` of the `roomkit-react` package.

```bash
"@tensorflow-models/toxicity":"^1.2.2"
```

It can be added using either of the following methods.

- Using `yarn`:

```bash
yarn add @tensorflow/tfjs @tensorflow-models/toxicity
```

- Using `npm`:

```bash
npm install @tensorflow/tfjs @tensorflow-models/toxicity
```

**Usage**

In the `ChatFooter.tsx` file, import the package as:

```jsx
import * as toxicity from '@tensorflow-models/toxicity';
```

We need to load the model

```jsx
const model = await toxicity.load(0.9, []);
```

Add the following after the message is taken from the input field to run it through the model:

```jsx
const predictions = await model.classify(message);
const isToxic = predictions.some(prediction => prediction.results[0].match);
if (isToxic) {
   ToastManager.addToast({
    title: 'Message contains toxic content',
   });
   return;
}    
```

<video loop="true" autoplay="autoPlay" id="vid" muted>
<source src="https://storage.googleapis.com/100ms-cms-qa/cms/tensorflow_moderation_demo_af9fd54ca2/tensorflow_moderation_demo_af9fd54ca2.mp4?updated_at=2024-04-09T10:11:21.420Z" type="video/mp4" />
</video>

Any toxic text now typed into the message field, would not be shared and the user notified of the same with a toast. Further action can be taken by notifying a moderator or changing the role of the user to prevent them from sending any further messages.

This is only scratching the surface of what is possible when it comes to chat moderation. In the next part, we will explore using a cloud service or a local LLM to moderate chat better. 

Until then, if you have any questions regarding 100ms, feel free to reach out to the team on our [Discord](https://discord.com/invite/kGdmszyzq2).

thumbnail_Tensor - 1.png

Tensor - 1.png

How to moderate Chat in 100ms? - Part 1

Pushpam Saxena

*Imagine this: you are lounging on your couch, savouring a moment of relaxation, when suddenly your phone comes alive with a familiar ringtone. With a casual tap, you're instantly transported, your buddy's face beaming through the screen as if they're right there with you, defying the boundaries of distance. It's a moment of magic, a seamless network connection that transcends physical barriers.*

However, have you ever wondered about the intricate mechanisms enabling these magical connections? Beneath the surface lies a sophisticated network facilitating real-time audio and video transmission, ensuring a smooth and uninterrupted experience.

This guide explains the behind-the-scenes of how to make a notification-based one-on-one call happen, while you video talk with your pal without dropping off the call.

---

## Introducing Blitz 💯

Blitz is a powerful one-to-one call application powered by [100ms](https://www.100ms.live/), enabling users to make video or audio calls with a range of key features such as:

- **Call List UI** - A user-friendly interface that displays a list of available contacts or recent calls, allowing users to initiate calls seamlessly.
- **Self-Preview** - During video calls, users can preview their video feed, ensuring they are ready and presentable before joining the call.
- **Call Options** - Users can customise their call experience options like enabling/disabling audio or video, sharing their screen, and flipping the camera between front and rear-facing modes.

- **Call Notifications** - When a user receives an incoming call, Blitz provides a notification, allowing them to accept or reject the call even if the app is not in the foreground.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Blitz_4_541e9aabc3/Blitz_4_541e9aabc3.png) 

## How did we make Blitz?

Blitz relies on key technologies:

- **100ms Template** - Template is the blueprint of the room. It defines the settings of the room along with the behavior of users who are part of it
- **Firebase** - Enables push notifications for incoming calls, even when the app is not in the foreground.
- **CallKit** - It provides functionality to show native call UI and handle the call session.
- **100ms Prebuilt** - Accelerates development using Prebuilt's open-source code, we’ve listed the UI customisations to power one-on-one calling functionalities.

Together these technologies power our Blitz app, enabling seamless one-on-one video calls.  Without further ado, let's dive in! 🚀



### Set up the 100ms Template for Audio and Video Calling

We’ve set up both options where the caller can make a video or audio-only call. For this, we’ll create a 100ms account, and set up below templates:

- **Audio Room Template** - In case you want to have only Audio-only calls
- **Video Conferencing Template** - In case you want Video Calls, with audio-only support.

Ensure two roles are `caller` and `callee` with permissions to `publish` and `subscribe` to each other. Also, ensure both roles have `end room` and `room state` permissions.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Blitz_2_1_bc11de9bef/Blitz_2_1_bc11de9bef.png) 
Since the `caller` waits in preview screen for the corresponding person to join, as soon as the other person joins, the `caller` gets `onRoomUpdate` with `peerCountUpdated`. 
To receive this alert, a setting has to be enabled on the 100ms Dashboard in the Advanced Settings section.

On receiving the notification, `Callee` has the complete Room information and can straightaway `join` the room. We achieve this using the `skipPreview` option in `HMSPrebuilt`. This option can be enabled from Customize Prebuilt section:

<video loop="true" autoplay="autoPlay" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/skip_preview_dashboard_602a9847f1/skip_preview_dashboard_602a9847f1.mov?updated_at=2024-04-09T08:08:52.611Z" type="video/mp4" />
</video>

### Let's Integrate Firebase

Firebase serves as the backend of our application, providing the necessary framework for initiating, transporting and handling required information for the calls. We have broken this into three steps:

1. **Call Initiation**: This is the first step in the calling process. The caller initiates a call to another person. A 100ms Room is created in this step using a Firebase function. This function returns the Room-Code for the corresponding Room.
2. **Sending Notification**: Once we have the Room-Code, the next step is to send this info to the `callee`. We leverage Firebase's Messaging capabilities to deliver this information. This involves sending a Push Notification to the receiver with the Room details. We have the receiver device details - the FCM token - in our database.
3. **Receiving Notification**: Firebase ensures that incoming calls are seamlessly handled, regardless of the recipient's app status, whether the app is actively running, in the background, or even closed. We use Firebase functions to show the call interface based on the app's state.

> Please note that detailing the process of integrating Firebase into the project falls outside the scope of this blog.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/diagram_export_03_04_2024_13_10_50_e303426cc2/diagram_export_03_04_2024_13_10_50_e303426cc2.png) 

### It's Time To Add CallKit

**[Callkit](https://developer.apple.com/documentation/callkit/)** gives us handy tools to ensure our call feels like a regular phone call with native UI. With CallKit, we can do all sorts of cool stuff, like displaying the standard call screen you're used to seeing, keeping track of missed calls, and even giving you the option to put calls on hold when you need a moment. 

For Callkit we are using the [flutter_callkit_incoming](https://pub.dev/packages/flutter_callkit_incoming) package. We leverage it’s APIs as shown below -

- **Caller initiates the call** - When the `caller` initiates a call, we use CallKit's **`startCall`** function to kickstart the call session. This ensures that if another call comes in, CallKit smoothly takes over, allowing options like putting the current call on hold and accepting the new one.
    
- **Callee receives a Firebase notification with Room information** - Upon receiving the Firebase notification, we trigger CallKit's `showCallkitIncoming` function. This action promptly presents the system call interface, complete with a ringer tone and other essential elements, ensuring a native call experience for the recipient.
    
- **Call Termination** - Upon call termination, we promptly execute the **`endCall`** function. This action effectively concludes the call session, ensuring the device exits the call state seamlessly.
    

### **Blitz Leverages 100ms Prebuilt Open Source UI**
The Blitz one-on-one app utilizes a forked version of the `hms_room_kit` package from 100ms Prebuilt's open-source [100ms-flutter](https://github.com/100mslive/100ms-flutter) repository. Leveraging Prebuilt's UI component accelerates development while providing a robust, feature-rich user interface tailored for audio/video calling across platforms.

Blitz brings a sleek and intuitive user interface powered by the [100ms-flutter-callkit](https://github.com/Decoder07/100ms-flutter-callkit) repository. This repository, derived from `100ms-flutter`, is tailored specifically for Blitz like audio/video calling apps, incorporating UI enhancements to elevate your experience.
![](https://storage.googleapis.com/100ms-cms-qa/cms/Blitz_3_1_23595d7a7a/Blitz_3_1_23595d7a7a.png) 
The prebuilt UI has been tailored for 1:1 calling on Blitz UI:
- Modified header and footer
- Removed peer names
- Added camera flip on local tile
- Removed chat and other options

These changes streamline the UI for a focused 1:1 calling experience, removing distractions and optimizing the core video communication functionality. Check out the complete changes [here](https://github.com/100mslive/100ms-flutter/compare/main...Decoder07:100ms-flutter-callkit:main).


 To use `hms_room_kit` package from this repo, add this to your `pubspec.yaml` file:

```kotlin
  hms_room_kit:
    git:
      url: https://github.com/Decoder07/100ms-flutter-callkit.git
      ref: main
      path: packages/hms_room_kit
```



## Bringing It All Together

1. Clone the repository
    
    ```bash
    git clone https://github.com/100mslive/100ms-flutter.git
    ```
    
2. Navigate to the project directory
    
    ```bash
    cd 100ms-flutter/sample apps/one_to_one_callkit
    ```
    
3. Install dependencies
    
    ```bash
    flutter pub get
    ```
    

> Note 🔑: The app utilizes the `callkit-ui` branch from the `hms_room_kit` repository instead of the published package to integrate a customized calling interface.


4. Set up Firebase for Push Notifications, Google Sign-in, and Cloud functions. Add `GoogleService-Info.plist` to the `ios/Runner` directory and `google-services.json` to `anroid/app` directory.
5. Create a 100ms account, and create templates as mentioned above.
6. Add the following fields in the below files:
    1. Update `CFBundleURLSchemes` in `Info.plist` with your client ID from `GoogleService-Info.plist`
    2. Update management token from 100ms dashboard, `Developer section` in `functions/index.js`
    3. Add Audio and video template IDs in `createRoom` method in `lib/services/app_utilities.dart`
    4. Add authentication client id in `loginUser` method from `GoogleService-Info.plist` file in `lib/services/app_utilities.dart`
7. Deploy cloud functions
    
    ```bash
    firebase deploy --only functions
    ```
    

That's it! You're ready to run the application.

### Test the application

```bash
flutter run
```

## Let’s make a call

Let’s verify that all components are properly configured and functioning to test the application.

1. **Install the App on Two Devices**: You need two to make and receive a call: a `caller` and the `callee`— log in using Google sign-in from both devices.
2. **Initiate the Call**: Initiate audio or video call to another device
3. **Answer the Call**: Once you get the call notification on the other device, tap the 'accept' button and voilà! We're officially chatting away!”

<video loop="true" autoplay="autoPlay" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/blitz_1_f42c6061c9/blitz_1_f42c6061c9.mp4?updated_at=2024-04-08T13:01:05.134Z" type="video/mp4" />
</video>

Building calling functionality is easier than it seems, thanks to the power of 100ms and Firebase. Explore this guide to see how straightforward the implementation can be. If you have any questions, our [Discord community](https://100ms.live/discord) is here to help.

thumbnail_Flutter 1-1.png

Flutter 1-1.png

Experience Blitz: One-to-One Video Calls Powered by 100ms and Firebase

We're excited to partner with Krisp, a cutting-edge AI audio technology that delivers superior noise cancellation. By integrating Krisp into our 100ms platform, we're enhancing the audio experience for our users, ensuring crystal-clear communication regardless of their environment.

Krisp is a revolutionary audio solution that leverages AI to eliminate unwanted background noise and voices, allowing only the primary speaker's voice to be heard clearly on calls. Whether you're working from a home office, a bustling co-working space, or even a noisy cafe, Krisp's Background Voice Cancellation feature seamlessly filters out ambient sounds, resulting in more efficient and professional communication.

## Cancel more than just noise

With Krisp, your audio quality is as crisp as a fresh potato chip, no matter how loud the surrounding racket. But noise cancellation isn't just a cool audio trick – it's a productivity powerhouse for audio/video calls. By combining Krisp's noise-zapping capabilities with 100ms, you unlock:

**Productivity Powerhouse**<br>
- 🧠 Laser-Sharp Focus: Background noise can shake your concentration faster than a jackhammer. Noise cancellation helps you tune out those distractions, so you stay engaged and productive.
 - 🗣️ Crystal-Clear Communication: Struggling to understand muffled voices is frustrating. Noise cancellation ensures every word comes through loud and clear, preventing misunderstandings and boosting productivity.
 - 🔇 Goodbye Mute/Unmute Hassle: Tired of that mute/unmute dance every time a noise pops up? Noise cancellation ends that productivity-killing routine.
 - No More Listener Fatigue: Constantly straining to hear over noise is exhausting. Noise cancellation provides a comfortable, fatigue-free audio experience.

**Professional Flexibility**
- 💼 A Professional Edge: Ambient noise can quickly turn a polished call into a messy situation. Noise cancellation helps you sound sharp, focused, and in-control.
- 🌍 Work from Anywhere, Hassle-Free: Whether you're calling from a noisy cafe or a park, noise cancellation allows productive conversations without sound interruptions.
- 🤝 Better Collaboration: Background noise makes it hard to share ideas and engage. By eliminating distractions, noise cancellation creates a more immersive, collaborative space.

## Hear the difference

Want to experience the power of noise cancellation for yourself? Take a demo on 100ms Prebuilt and see Krisp in action on 100ms. 

<add video>

## Noise cancellation with Krisp to your 100ms sessions

### Embed Krisp using 100ms SDKs

Integrating Krisp into your 100ms applications is a seamless process, allowing you to harness the power of noise cancellation with minimal effort. Here's how you can enable Krisp on our various SDKs:
- **[Web ](https://www.100ms.live/docs/javascript/v2/how-to-guides/extend-capabilities/plugins/krisp-noise-cancellation)**
- **[Android](https://www.100ms.live/docs/android/v2/how-to-guides/extend-capabilities/noise-cancellation)**
- **[iOS]()** 


### Enable Krisp on 100ms Prebuilt

100ms Prebuilt is a feature rich live video app that can be embedded in your web or mobile apps. It comes with built-in noise cancellation support powered via Krisp. By default, it is turned off, however users can enable it on preview screen prior to joining their 100ms room or they can enable it under microphone options.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Krisp_3_216fb81d70/Krisp_3_216fb81d70.png) 

### Cross platform support with 100ms Prebuilt

100ms Prebuilt can be used as links which can be shared or embedded as iframe. Alternatively, it can be embedded as a native component on Web, iOS, Android, Flutter and React Native. You can demo noise cancellation with 100ms Prebuilt using

- **[100ms Dashboard](https://dashboard.100ms.live/dashboard)** - join a room (Prebuilt links); supported on Chromium based browsers such as Chrome, Microsoft Edge.
- **100ms Apps**
    - Get native **[Android app here](https://github.com/100mslive/AndroidPrebuiltDemo/commit/7e4150df9b708d63b2e678ce8fc6aa81f64bdd6c/checks)**
    - Get native **[iOS app here](https://testflight.apple.com/join/Pb4aTS0t)**

Support for Krisp on 100ms Prebuilt for Flutter and React Native, will be coming soon.

## Get started today

To enable Noise Cancellation via Krisp on 100ms, reach out to us on [support@100ms.live](mailto:support@100ms.live)

thumbnail_Krisp - 2.png

Krisp - 2.png

Reduce noise with Krisp on 100ms

100ms team

WebRTC powers most live video conferencing tools you see - Google Meet, Teams, your telehealth provider’s app, and the live spaces in your favorite social media app. WebRTC has allowed video conferencing to move from closed expensive systems to browsers and devices everywhere. But for developers building with WebRTC, debugging calls issues like “Can’t hear you”, “You broke up there for a while” isn’t as straightforward. This is especially true during rollout.

Developers often go live with settings that might not be optimized for their use case. End-users don’t always have the same (often great) network conditions/devices used during development. And simulating poor network conditions for WebRTC isn’t a part of most front-end developers’ toolkit. E.g. a telehealth app’s end-users can sometimes have;

- Patchy WiFi setups (routers placed far away from rooms), with above-average packet drops. 
- Older Safari versions can have bugs that cause echo during video calls. 

These are specific issues but they can extend to other devices and use cases. This needs testing outside of office WiFis, simulated with UDP packet losses, and older iPhones.

*This is part 1 of our series on how 100ms’ enterprise support adds dogfooding and early check-ins to help developers go live with minimal errors.*

At 100ms, we bridge this gap with our premium and enterprise support packages by offering dogfooding sessions. This involves 100ms’ customer success team running tests on the customer’s app before rollout.

As part of this, 100ms’ customer success manager understands the end-users’ setup and designs test cases. These cases are run on the customer’s beta app on 100ms’ device farm and run through our setup to simulate stressful network conditions - higher **RTT** (Round trip time), packet losses, or throttled upload & download bitrates. During these tests, 100ms runs through multiple settings, and then recommends the optimum settings to the customer.

 This testing often saves the customer 4-6 weeks of iteration with real users to get the same settings.

This blog outlines the impact of these dogfooding sessions with a real-world case study.

### Case study

Let’s first understand quickly what parameters a developer can set when integrating a WebRTC application. 

Developers have the following 100ms settings to play with (and for a few other WebRTC APIs)

- **Highest publish layer settings** - The ideal resolution, fps, and bitrate at which each user will publish their videos. E.g. 720p, 30 fps, 850 kbps.

- **Simulcast settings** - The publishing user can publish up to 3 layers of videos (540p/30fps/480kbps, 360p/30fps/320kbps) with subscribers moving to lower layers in poor network/device conditions

- **Subscribe degradation -** This is a special case of fallback implemented by 100ms, where the subscribing user subscribes only to the audio, in case there’s no bitrate available for the video to be subscribed to, even at the lowest simulcast layer.

![WebRTC settings on 100ms dashboard](https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_d3336cd5ff/Untitled_d3336cd5ff.png?updated_at=2024-03-27T07:41:57.385Z)

In this case, the customer is a dating app where users can have 1-on-1 audio or video calls. Most customers are in urban US/EU regions.  

The developers chose the following settings for their app

- **Highest publish layer settings** : 720p, 30fps at 900 Kbps

- **Simulcast settings** :

| | Resolution | Frame Rate | Max Bitrate |
|:---:|:---:|:---:|:---:|
| Layer 1 (Highest publish layer setting) | 720p | 30 fps | 900 kbps |
| Layer 2 | 360p | 30 fps | 500 kbps |
| Layer 3 | 180p | 30 fps | 200 kbps |

- **Subscribe Degradation** : Enabled

These are standard settings, and what we often see developers go live with. After getting access to the beta build, 100ms’ customer success team worked with the developers to design test cases for their scenarios.

### First dogfooding session results

While dogfooding the app, 100ms’ customer success team identified a problem where the video resolution kept dropping frequently and the video would often get pixelated. This happened specifically in slightly older Android/iOS phones. These phones represented a double-digit percentage of the app’s user’s devices.
<center>
<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted style="width:30% !important;">
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_4ec750ce0b/Untitled_4ec750ce0b.mp4?updated_at=2024-03-27T07:49:59.831Z" type="video/mp4" />
</video></center>

### Debugging

To help isolate these issues, we validate the dogfooding test results with data - WebRTC stats. These stats keep track of video resolution, bitrate, frame rate, RTT (Round Trip Time), etc. that every user is publishing at any point in time. 

Looking at the publish resolution data from publisher stats from the problematic sessions, it was clear that the user was only able to publish the 2nd simulcast layer of 360p but not the primary 720p layer. All other output results seemed fine.

![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2024_01_30_at_2_13_45_PM_1_2bf798a9c9/Screenshot_2024_01_30_at_2_13_45_PM_1_2bf798a9c9.png?updated_at=2024-03-28T21:16:45.860Z)

There could be one of 2 factors contributing to the lower resolution - internet connectivity or device CPU limitations. In this case, the `average_outgoing_bitrate` which is the best estimate of network being of poor health. When we looked at `publisher_quality_limitation` which is one of the publisher stats, it was clear that the device was unable to publish the 720p video because of CPU limitations. This confirmed the eye-test that this was not because of network conditions, but due to these older phones not being able to consistently publish video at the highest publish layer.

![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2024_01_24_at_4_50_22_PM_1_0d68cbd1ea/Screenshot_2024_01_24_at_4_50_22_PM_1_0d68cbd1ea.png?updated_at=2024-03-29T20:22:37.445Z)

### Solving the Problem

It's surprising how many video conferencing tools (including Zoom, Google Meet) will often default to slightly lower framerate, without users noticing the difference in 1-in-1 calls. 720p with 30fps turns out to be too much for even slightly older devices. This causes devices to default to the next layer - 360p, which is visibly worse. 

We wanted to reduce the CPU load on these low-end devices but still retain the quality. We ran the next dogfooding session by changing the publish layer settings to the following:

|  | Publish resolutions | Frame rate | Max Bitrate |
|:---:|:---:|:---:|:---:|
| Layer 1 | 540p | 15 fps | 1100 kbps |
| Layer 2 | 480p | 30 fps | 1000 kbps |
| Layer 3 | 480p | 15 fps | 800 kbps |

We reduce CPU load by 2 ways here.

- Reducing the highest layer from 720p to 540p
- Keeping the last 2 layers at the same resolution.  For both these layers, the publisher effectively needs to publish just once - at 480p/30fps. Switching from layer 2 to layer 3 can happen between the server (SFU) and the subscriber.

### Second dogfooding session results

Here’s a before and after comparison of the video quality after changing the settings.

**<center>Before</center>** 
<center>
<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted style="width:30% !important;">
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_1_2fa7a2077f/Untitled_1_2fa7a2077f.mp4?updated_at=2024-03-27T10:15:53.535Z" />
</video></center>

**<center>After</center>**
<center>
<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted style="width:30% !important;" >
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_2_94a46417a8/Untitled_2_94a46417a8.mp4?updated_at=2024-03-27T10:15:54.135Z" />
</video></center>

With the same device and network conditions, these settings look much better in the older phones we saw the issue in. And for newer devices, this

- A higher frame rate (fps), such as 30 fps, does not always result in better video quality, especially on mobile devices. A lower fps rate, like 15 fps, can perform better without a noticeable visual difference.
- 540p at 1100 kbps yields excellent quality with sharp video, functioning well across various high-end and low-end devices.
- Reducing simulcast layers to two can further reduce the load on the device, improving video quality.

### Conclusion

Understanding the end users’ device, location, and internet quality demographics is crucial to tailoring an optimal experience. There are no one-size-fits-all template settings that will solve for all use cases. We’ve thoroughly tested across a range of devices, a variety of internet conditions, and browsers, and have also set up periodic automated tests for the same purpose. This comprehensive testing gives us the advantage when we’re testing our customers’ setup. Our experience not only lessens your overall testing load but also allows you to arrive at the optimal settings faster.

Because issues often affect a minority of users, it is easy to go live and not fix these issues immediately. Dogfooding apps in networks/devices close to the end-users' settings save several weeks of poor user experience and ratings.

---


thumbnail_Testing - 3.png

Testing - 3.png

Optimizing Video Call Quality: Lessons from 100ms' Dogfooding and Testing

Picture this: you effortlessly kickstart interactive polls directly from the whiteboard content in just a few clicks. 

We're excited to release “Polls AI”, a demo built on top of [tldraw’s](https://www.tldraw.com/) [Make Real template](https://github.com/tldraw/make-real-starter) with live audio-video powered by 100ms, it uses OpenAI’s GPT Vision to analyse the shapes, conjures up the perfect question, and crafts options that launch polls in the blink of an eye, captivating your audience like never before! Amazingly, we live in a world where the new-age GPTs can actually power that. 

<iframe width="100%" height="380" src="https://www.youtube.com/embed/I2SlAEXnaTo" title="Whiteboard to Polls in 1 click - Polls AI Demo" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

<br />

With Polls AI, educators can now launch multiple-option polls to evaluate the understanding of the participants in a single click - no more interruptions in the flow. No more trying to come up with random options to add to the polls. Polls AI does that for you!


## What are the advantages?

With 100ms’ live audio-video calling and GPT Vision, educators can now create polls effortlessly by selecting the content on the tldraw whiteboard. This not only fosters engagement but also allows tutors to continue with their lectures without constant interruptions.

This allows for:

- real-time Interaction, as the teachers can create polls on the fly during live classes which ensures that the learning experience remains dynamic and engaging.
- greater efficiency and time savings, as educators can focus more on delivering content and less on administrative tasks.
- more spontaneity in assessment, as polls, can be quick and spontaneous, enhancing the learning environment and encouraging students to contribute to class discussions actively.

## Where can this be used?

Imagine an ongoing virtual classroom. The teacher is in the middle of the lecture explaining the concepts using the whiteboard. They have been speaking for a long and want to see if the students have grasped the concept or not. Instead of having to launch a side pane, and fill out a questionnaire to add questions and some random options to the poll, they can simply slide the cursor across the shapes and click on the ‘Create Poll’ button.

AI recognises the whiteboard content and creates a question for the poll and also comes up with the options as suitable. The teacher can quickly check if the questions and options are correct, edit them if required and finally launch the poll. 

<iframe width="100%" height="380" src="https://www.youtube.com/embed/V8aYe-SAjlw" title="Whiteboard to Polls in 1 click - Polls AI Demo" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

<br />
Once the poll is live, students can cast their votes, engaging them and also helping the teacher understand how well have they understood the lesson. All this without any precious time by harnessing the power of AI.

The demo can also be used to annotate images from one’s presentation and launch a poll on the same.

The integration of GPT Vision with live audio-video calling shows the transformative potential of AI in education technology. As we continue to explore innovative ways to enhance virtual classrooms, the integration of AI-driven solutions holds immense potential. 

> Our vision is to empower educators, enrich student experiences, and create a more seamless and interactive learning environment.

We are excited for educators everywhere to try this out!

Get started here: [make-real-polls.vercel.app](https://make-real-polls.vercel.app/)

We would love to continuously improve this demo based on user feedback. As educators and students engage with the Polls AI, their insights and experiences will shape future iterations.

thumbnail_Polls AI Banner.jpg

Polls AI Banner.jpg

Polls AI: Whiteboard to Polls in an instant!

We're excited to share that 100ms Prebuilt UI is now completely open-source.

On the web, we have made the monorepo that contains `roomkit-react` and `roomkit-web` (which contain the Prebuilt component and the web component port of the `HMSPrebuilt` component) public and freely available to use under the MIT License.

> You can find all of the code on [GitHub](https://github.com/100mslive/web-sdks).

On other platforms, the code for Prebuilt is available at the following links with READMEs to guide you through the integration process and provide examples of common use cases.

| Client | Repository | Documentation | Example |
|--|--|--|--|
| Android | [100ms-android](https://github.com/100mslive/100ms-android/tree/release-v2/room-kit) | [Link](https://www.100ms.live/docs/android/v2/quickstart/prebuilt-android) | [AndroidPrebuiltDemo](https://github.com/100mslive/AndroidPrebuiltDemo)
| iOS | [100ms-roomkit-ios](https://github.com/100mslive/100ms-roomkit-ios) | [Link](https://www.100ms.live/docs/ios/v2/quickstart/prebuilt) | [100ms-roomkit-example](https://github.com/100mslive/100ms-roomkit-example)
| Flutter | [100ms-flutter](https://github.com/100mslive/100ms-flutter/tree/main/packages/hms_room_kit)| [Link](https://www.100ms.live/docs/flutter/v2/quickstart/prebuilt) | [hms_room_kit/example](https://github.com/100mslive/100ms-flutter/tree/main/packages/hms_room_kit/example)
| React Native | [100ms-react-native](https://github.com/100mslive/100ms-react-native/tree/main/packages/react-native-room-kit)| [Link](https://www.100ms.live/docs/react-native/v2/quickstart/prebuilt) | [react-native-room-kit/example](https://github.com/100mslive/100ms-react-native/tree/main/packages/react-native-room-kit/example)

## Why are we doing this?

Integrating live audio-video capabilities into applications presents unique challenges. How do you create dynamic video layouts? Manage participant lists? Incorporate features like chat, hand-raising, and more? At 100ms, we aim to simplify these complexities and empower developers to build exceptional real-time audio-video experiences.

- **Simplifying for out-of-the-box experiences with 100ms Prebuilt** - The Prebuilt UI offers essential features like chat, participant lists, emojis, and more, ensuring a consistent user experience across various screens, including preview and room views. By abstracting away the complexities involved in building real-time audio-video experiences, Prebuilt streamlines the development process, accelerating time-to-market for developers.
- **Open sourcing rationale** - By abstracting away complexities, Prebuilt accelerates development cycles and time-to-market for real-time audio-video experiences. As we build video infrastructure for a video-first world, we recognize the crucial role of the global developer community. Making Prebuilt open-source aims to make it more accessible, transparent, and widely available.
- **Value for developers** - Developers can now examine the inner workings of our SDK, spot potential issues, and suggest improvements. This collaborative approach fosters innovation and continuous improvement in a way that is not possible in a closed system. We are excited about the opportunities this change brings and remain committed to building easy-to-integrate SDKs that empower developers to create exceptional real-time audio-video experiences.

#### Plus, a lot more flexibility!

While we offer [a no-code way to customize Prebuilt](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/overview#customize-prebuilt), you can now fork your copy of the Prebuilt component (on any platform of choice) and make changes to the code to allow for more fine-tuning. 

Say you wanted to customise the background on the Leave Screen on Prebuilt React, to something more unique to your organisation, which by default looks as below: 

 ![Original Leave Screen of 100ms Prebuilt in the open source web sdks](https://storage.googleapis.com/100ms-cms-qa/cms/1_22fd4180af/1_22fd4180af.png) 

You can now fork the repository [here](https://github.com/100mslive/web-sdks). Clone it to your system and make changes to the code. A quick search on the IDE would reveal the code for Leave Screen is present in the packages folder under Prebuilt components with the path as `packages/roomkit-react/src/Prebuilt/components/LeaveScreen.tsx` 

I’ll quickly modify the emojis and change some text for this demo. And, when ready with the changes, run `yarn build` to build the packages. You can now navigate to `prebuilt-react-integration` inside `examples` to test the changes. Run `yarn` and `yarn dev` to view the changes made to the Leave Screen.

 ![Modified Leave Screen of 100ms Prebuilt using open source web sdks](https://storage.googleapis.com/100ms-cms-qa/cms/2_6d41f2e976/2_6d41f2e976.png) 

This is only a fraction of what is possible with open-source Web SDKs. 

Similarly, find guides to modify Prebuilt on iOS [here](https://github.com/100mslive/100ms-roomkit-ios?tab=readme-ov-file#how-to-make-changes-debug-preview-and-contribute-to-roomkit), on Android [here](https://github.com/100mslive/100ms-android?tab=readme-ov-file#%EF%B8%8F-developing-against-prebuilt-room-kit), on Flutter [here](https://github.com/100mslive/100ms-flutter/tree/main/packages/hms_room_kit), and on React Native [here](https://github.com/100mslive/100ms-react-native/tree/main/packages/react-native-room-kit). 

## What’s next?

We welcome your feedback and would happily help external contributors or anyone excited to contribute. If you'd like to get involved, check out our [contribution guide](https://github.com/100mslive/web-sdks/blob/main/DEVELOPER.MD) and explore the codebase. See you there!

Try 100ms today. [Get started here!](https://www.100ms.live/docs)
Have any questions? Reach out to the 100ms team on [Discord](https://discord.com/invite/kGdmszyzq2).

thumbnail_Prebuilt OS (1).png

Prebuilt OS (1).png

100ms Prebuilt is now Open Source! 

Mantra Manan Saraswat

In today's digital age, where personal information is constantly exchanged and stored online, the security and confidentiality of health data have never been more critical. From medical records to insurance details, our health information is very personal and needs the highest level of protection. Enter the Health Insurance Portability and Accountability Act, commonly known as HIPAA. This landmark legislation, enacted in 1996, revolutionized the way health information is handled across healthcare and insurance industries.

Before delving into the intricacies of HIPAA, it's essential to understand the landscape that made it critical. The late 20th century marked a pivotal shift in how health information was managed, moving from paper records to an increasing reliance on electronic systems. This transition brought with it a host of new challenges and vulnerabilities – from data breaches to unauthorized access – making the protection of sensitive health information a pressing concern. HIPAA was born out of this need, aiming to strike a balance between the efficient handling of health data and the privacy rights of individuals.

Although HIPAA is a federal law with a set of guidelines on how protected health information or PHI must be handled, the interpretation and implementation of this law will differ from organization to organization within the context of their services or product. This makes it complex to implement especially considering there are plethora of ways where ePHI is created, transmitted, transformed or stored within any tech product or service.

In this blog, we will give a primer on what [HIPAA compliance](https://www.hhs.gov/hipaa/for-professionals/index.html) is, who it applies to, what are the key guiding principles, and how it applies to the new age tech companies.

## Who does it apply to?

Department of Health and Human Services (Referred to as HHS henceforth) has [explained HIPAA’s scope quite succinctly](https://www.hhs.gov/hipaa/for-professionals/covered-entities/index.html) but broadly. 

**Covered Entities**

Organizations or individuals offering health plans, healthcare providers (doctors, clinics, pharmacies, etc), and healthcare clearing houses. These are the primary entities responsible for maintaining and protecting PHI under HIPAA.

**Business Associates**

A business associate is a person or an entity that performs certain functions or activities involving the use or disclosure of PHI on behalf of, or provides services to, a covered entity.

**Subcontractors**

These are third-party entities that business associates engage with to carry out some of their services. A subcontractor is a business associate of a business associate and must comply with the same HIPAA requirements.  

The following diagram depicts the relationship between the different entities along with an example.

![HIPAA Scope](https://storage.googleapis.com/100ms-cms-prod/cms/hipaa_scope_e8a79a74fc/hipaa_scope_e8a79a74fc.png?updated_at=2024-02-12T23:11:02.161Z)

As we will see shortly, HIPAA applies to every one of these entities, and all of them are required to get into a HIPAA agreement with their respective business associates. More on the agreement later but there is one kind of entity in this which is not required to enter into an agreement.

There is an interesting exception within HIPAA called the **‘Conduit exception’** which excludes certain entities from being classified as Business Associates. Not all entities that come into contact with PHI are considered Business Associates. The conduit exception applies to organizations that act merely as conduits for the transport of PHI and do not access or view the information other than on a transient or incidental basis as necessary to perform the transportation service. Postal services or ISPs fall in this category. 

### The key guiding principles to understanding HIPAA

There are 3 key rules to understanding HIPAA.  They apply to covered entities or their business associates who have created, received, maintained, or transmitted any PHI.

1. [**Privacy Rule**](https://www.hhs.gov/hipaa/for-professionals/privacy/index.html)
    
    This rule is a cornerstone in protecting the privacy of individuals' medical records and other personal health information while allowing the flow of health data needed to provide and promote high-quality health care.
    
    1. **Protect PHI (Protected Health Information)** - The Privacy Rule protects all "individually identifiable health information" held or transmitted by covered entities. PHI could consist of medical records, recordings, transcripts, billing information, or any other information that could identify a patient and their associated health conditions.
    2. **Minimum Necessary Standard** - The rule requires that only the minimum necessary information be used or disclosed that is required for the purpose of the treatment.
    3. **Patient Rights Over Their Information** - Patients have rights over their health information, including the right to examine and obtain a copy of their health records and to request corrections.
    4. **Restrictions on Certain Disclosures**: The rule places restrictions on the use and disclosure of PHI for marketing, sales, and other purposes without explicit patient authorization. 
2. **[Security Rule](https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html)**
    
    This rule requires applicable entities to put in administrative, physical, and technical safeguards to protect from unauthorized access, breaches, and transmission security. This is to ensure the confidentiality, integrity, and availability of that information.
    
    1. **Administrative safeguards** would include policies on security officials, risk assessment and mitigation, workforce training, and information access management.
    2. **Physical safeguards** include policies and controls to protect data in case of environmental hazards and physical unauthorized intrusions. This could include device and media control of the workforce, security, and fail safes in case of possible breaches (like getting the device stolen).
    3. **Technical safeguards** include policies on access controls, audit controls and trails, and integrity controls (improper alteration or destruction.
    4. Risk analysis, monitoring, and periodic review along with having a policy on contingency planning is an important part of this rule.
3. **[Breach Notification Rule](https://www.hhs.gov/hipaa/for-professionals/breach-notification/index.html)** - In case of unsanctioned access or breach of ePHI, the covered entities, and their business associates are required to notify all the affected parties, HHS, and in some cases, media if the breach is big enough (affecting more than 500 people). This notification usually also includes mitigation measures that have been taken already and additionally, need to be taken by the affected parties to safeguard themselves from further harm and breaches. This notification must be released within 60 days of the breach.

In addition to these, we also have two more recent additions to these rules which are an extension;

1. **Enforcement Rule**: Specifically, this rule gives the Office for Civil Rights (OCR) within the US Dept. HHS to conduct investigations and compliance reviews of covered entities as well as their business associates. This lays down when can these reviews take place, the method of complaint, categories of violations and associated penalties, and redressal mechanisms.
2. **Omnibus rule:** These are a set of rules introduced to extend HIPAA’s reach and increase overall accountability for handling confidential data.
    1. Previously, business associates weren’t directly liable under HIPAA but this ruling directly makes them accountable.
    2. Previously, notification was required if the breach posed a "significant risk of harm" to the individual. The Omnibus Rule changed this to a presumption that any impermissible use or disclosure of protected health information (PHI) is a breach unless the covered entity or business associate demonstrates that there is a low probability that the PHI has been compromised.

Each rule under HIPAA not only details the responsibilities of the involved parties but also defines the hefty penalties that come along with their breach. Surprise assessments and security checks by the **Office of Civil Rights (OCR)** are a common occurrence and the fines can range from a couple hundred to millions of dollars. But the penalties aren’t limited to civil monetary fines, criminal proceedings can also be initiated which can have jail terms of up to 10 years in cases of negligence with malicious intent.

There is no official certification body for HIPAA. So the involved parties must make sure they are HIPAA compliant. To ensure that, most entities dealing with and exchanging sensitive health information enter into an agreement. These agreements are called Business Associate Agreements or BAAs. 

## What’s the significance of a BAA?

To outline the ways that a business associate complies with HIPAA, a Business Associate Agreement (BAA), covers the responsibilities and risks that the [business associate](https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/business-associates/index.html) is taking on.
A BAA is a contract between a HIPAA-covered entity and HIPAA business associates.

This agreement outlines how both parties are practicing compliance. BAAs include:

- The services provided by the business associate.
- Types of data they are interacting with and details on permitted usage of the PHI by the business associates
- Responsibilities of the mentioned parties to comply with HIPAA
- Compliance duties as guided by HIPAA privacy, security and breach notification rules
- Termination and indemnification points

Ultimately, a business associate agreement (BAA) is mandated in order to meet HIPAA requirements and ensure that data security standards are upheld and that data is stored and used appropriately. But how can a covered entity be sure that its data is completely safe and that proper accountability measures have been taken?

![HIPAA BAA](https://storage.googleapis.com/100ms-cms-prod/cms/hipaa_baa_fbd251d5d9/hipaa_baa_fbd251d5d9.png?updated_at=2024-02-12T23:11:03.417Z)

Each entity needs to sign a contract with the parties it is associated with in the layer above (providing services to) and below (getting services from) to self-regulate themselves. [See the chart above]

In cases where Business Associates themselves are using certain services that might access PHI, they are required to sign a BAA with the said subcontractor. Business Associate Sub-contractor Agreement (BASA) and SA (Sub-contractor Agreement) are just another form of BAA, but just between layers that are not directly interacting with a covered entity yet still receiving their PHI data in some form. But there are cases where BAA isn’t necessary. We have talked about it already - the conduit exception. 

Here’s a model [BAA](https://www.hhs.gov/sites/default/files/model-business-associate-agreement.pdf) provided by HHS for your reference.

## HIPAA in Tech

HIPAA isn’t easy to implement. A lot of vendors will claim they’re HIPAA compliant, which might be true, but they may very well put the onus of integration of their service along with implementing the HIPAA guidelines on you. It is very important to understand how any of your potential business associates have implemented HIPAA compliance within their systems. We compiled a couple of questions that can help you get started with evaluating a service’s compliance level. These are broad questions with some specific details related to telehealth.

1. Can you detail the types of Protected Health Information (PHI) your service encounters? How is this information processed, stored, and ultimately disposed of?
2. What specific uses do you have for the PHI you collect? Please provide examples of how this information is utilized within your service.
3. Regarding the electronic PHI (ePHI) generated by your service, could you explain how it is created, where it is stored, and the lifecycle of this data?
4. Please elaborate on the encryption methods used for securing media and ePHI both in transit and at rest. Additionally, specify the retention periods for data stored in your databases.
5. Do you have appropriate safeguards to protect the PHI that you’ve collected?
    1. What are administrative, physical, and technical safeguards put in place?
    2. Do you have access controls and audit trails within your critical services?
6. For any subcontractors or third-party services you engage with, who might have access to PHI, do you establish Business Associate Agreements (BAAs)? How do you ensure their compliance with HIPAA? What are these services?
7. Could you describe your data retention and deletion policies, especially as they pertain to PHI? How do you ensure that data is securely deleted?
8. Besides HIPAA, are you compliant with any other data protection regulations or do you hold any certifications that impact your data handling practices (e.g., SOC 2, GDPR)?
9. Where is the data physically stored? Please specify the regions or countries where our data will be hosted.
10. In the event of a data breach, what are your policies for notifying affected parties and regulatory bodies? What steps are taken to mitigate the damage?
11. What are the associated costs of enabling HIPAA compliance for our account and what is it going to cost us?

Wrapping up this primer on HIPAA compliance, it's clear that dealing with its complexities can be quite the task. But don't let the nitty-gritty of implementing these standards scare you off from using the services you need. It's really important to get a good grip on HIPAA – it helps you understand what's needed to make your systems compliant.

Here at 100ms, we've tackled the tough parts of HIPAA compliance so your integration is smooth and straightforward. We go the extra mile by turning away any PHI that's sent to us by mistake and setting up strong protections for the PHI that we do handle. If you're curious about how we've made HIPAA compliance a seamless part of what we offer, make sure that you’re subscribed to our blog and newsletter.

thumbnail_HIPAA.png

HIPAA.png

A Primer on HIPAA Compliance

Do you have a brilliant idea for a live video app, but don't want the headache of building it from scratch? Meet 100ms Prebuilt. This ready-made video conferencing solution packs everything you need into one slick package - no coding required. Read more about it here

But what if you still want to make it your own? That's where our Prebuilt Customizer comes in. With a few tweaks to the interface, some color changes, and enabling or disabling key features, you can craft a uniquely branded experience for your users - without writing a single line of code!

## 🤔 Why Should You Customize?

Building your video use-case required developer bandwidth and weeks of effort, with Prebuilt, you can skip that headache. You want to tweak the chat settings, play around with participant lists; it's all about making your app truly yours, while keeping the setup snappy. 🫰

### Enter the Prebuilt Customizer
Basic custom themes and layout adjustments are table stakes these days. But with Prebuilt Customizer, you get so much more:

- **Ease of Use**: Unlike other platforms that often require a steep learning curve, we provide a straightforward customization process that truly requires no prior coding experience.
- **Customisation Depth**: We offer a level of customization that goes beyond basic color changes and layouts. With 100ms, every aspect of the UI and UX can be tailored to fit your brand identity and user needs.
- **Live Preview**: Real-time preview isn't just a feature—it's a game changer. This allows for immediate feedback. No more guesswork!
- **Packed with Features**: Set up options like 'Be Right Back', emoji reactions for interactive sessions, and controls for bringing others on stage..

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_1_dc8b156839/Untitled_1_dc8b156839.png) 

## No-code, but better

Ready to play around with the Prebuilt customizer? Here’s a sneak peek at what you can do:

### 🎨 Appearance Customization

- **Logo**: Make your brand unmissable by adding your own logo to the interface.
- **Theme Type**: Set the vibe with Dark or Light themes.
- **Color Palette**: Match your brand's color scheme by adjusting primary, secondary, and background colors. Offer you a cohesive design whether you choose the light or dark side.

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/appearance_3f902d4331/appearance_3f902d4331.mp4?updated_at=2024-01-15T13:28:33.236Z" type="video/mp4" />
</video>


### ⚙️ Screens and Components Customization

- **Preview Screen**: Customize the welcome screen that users see first. Here, you can add titles, subtitles, update the join button text and enable/disable virtual background - with custom background.
- **Room Screen**: Adjust the layout, enable features like chat, participant lists, emoji reactions, bring on stage and more to suit your needs.

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/screens_dad7712181/screens_dad7712181.mp4?updated_at=2024-01-15T14:24:15.830Z" type="video/mp4" />
</video>

### 🎛️ Detailing the components

- **Chat**: Decide whether to have a chat component, customize its initial state, and overlay view.
- **Participant List**: Choose to display or hide the list of participants in the room.
- **Video Tile Layout**: Control how video feeds are displayed, including the spotlighting of certain participants and layout adjustments.
- **Additional Features**: Set up options like 'Be Right Back' screens, emoji reactions for interactive sessions, and controls for bringing others on stage.

### 🔄 Live Preview for Real-Time Adjustments

As you tweak settings, watch your changes come to life on the spot.

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_978d40f8bd/Untitled_978d40f8bd.gif?updated_at=2024-01-15T14:25:09.087Z" type="video/gif" />
</video>


##  Try it out

- Sign in or create an account at [dashboard.100ms.live](http://dashboard.100ms.live)
- Choose a template or make a new one.
- Click on “Customize” to use the Prebuilt Customizer
- Revisit the section above to learn about customization

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Customiser_video_0a61d79839/Customiser_video_0a61d79839.mp4?updated_at=2024-01-15T14:25:09.390Z" type="video/mp4" />
</video>



## Customize Away

The 100ms Prebuilt Customizer is your key to creating unique video and audio experiences tailored just for you. It's easy to use and packed with powerful features, all aimed at making your app as special as your idea. Say no to boring, one-size-fits-all solutions.

And this is just the beginning! Stay tuned for more updates, and join our [#Prebuilt Discord channel](https://discord.gg/jD94Fp74Ea) to be part of our awesome community.


### Related Resources
Check out our docs for more details:

1. [Prebuilt Overview](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/overview)
2. [Prebuilt Quickstart Guide](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/quickstart)
3. [Customize Appearance](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/Appearance)
4. [Customize Screens and Components](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/Screens-and-components)

thumbnail_Customiser.png

Customiser.png

Customize 100ms Prebuilt With No-Code

### Background

Superset, a dynamic placement automation platform, connects students, recruiters, and educational institutions to streamline the recruitment process. With a vast network of premier institutes like IITs and top-tier companies like Cognizant, PwC, Capgemini, and Wipro, Superset facilitates the entire job-seeking journey from application to acceptance to joining formalities.

### The Challenge

Initially, Superset used Twilio Video for their critical interview module, allowing recruiters and candidates to interact seamlessly. However, they needed a more reliable and efficient solution due to specific challenges around network connectivity and billing. Under challenging network conditions, their setup would consistently cause issues for both the interviewers and interviewees. At times, they were even forced to take the calls outside the platform. Twilio’s sample app was configured badly, and wasn’t working optimally for bad network conditions. Moreover, their support was less hands-on in helping resolve these issues. For Superset, every interview was important and a single bad experience was detrimental to both the interviewer as well as interviewee.


### Transition to 100ms

In March 2023, Superset began exploring alternatives to Twilio Video, and 100ms emerged as a promising candidate. The switch was driven by the need for a more cost-effective, reliable solution with a better technical support. 

### Integration and Usage

Superset's integration with 100ms was smooth and efficient. The platform's versatility accommodated different interview formats, from one-on-one to panel interviews, across various institutes and companies. 100ms's robust architecture ensured minimal network issues, a critical requirement for Superset’s global outreach.

![superset quote](https://storage.googleapis.com/100ms-cms-prod/cms/Superset_quote_a7d1d707ac/Superset_quote_a7d1d707ac.png?updated_at=2024-01-04T07:51:03.915Z)

### Features and Benefits

- **High Reliability:** 100ms provided a stable and reliable platform for conducting interviews, a crucial aspect for Superset's operations.
- **Enhanced Interview Experience:** The platform supported features like **screen sharing** and **whiteboard**, enhancing the interview experience for both candidates and recruiters.
- **Fraud Prevention:** Superset built sophisticated photo and face matching technology on top of 100ms by extracting associated video stream and capturing images to ensure the authenticity of candidates participating in interviews.
- **Flexibility:** The platform's adaptability allowed for a variety of interview formats, catering to diverse recruitment needs. They wanted a solution that support various interview formats, such as one-on-one, panel, and group discussions. All of these were supported and powered by 100ms’ templates.

### Results and Future Prospects

Since integrating with 100ms, Superset has seen a noticeable improvement in the efficiency and reliability of their interview process. The platform's robust feature set and stable performance have been instrumental in conducting thousands of interviews seamlessly. Superset rates their experience with 100ms as very positive, acknowledging the scope for further enhancements and expressing eagerness for future collaborations to refine the recruitment experience further.

### Conclusion

Superset's transition to 100ms exemplifies the platform's capability to handle large-scale, varied recruitment needs with efficiency and reliability. This partnership highlights 100ms's commitment to providing solutions that cater to the unique challenges of modern recruitment processes.

We recently wrote a blog on [comparing quality of 1-on-1 calls on 100ms and Twilio](https://www.100ms.live/blog/twilio-vs-100ms-call-quality-comparison).

thumbnail_Superset case study cover.png

Superset case study cover.png

Superset's Seamless Transition to 100ms for Enhanced Interview Experiences

## Use case: 1:1 Call Quality

As Twilio announces the end of life for their Programmable Video product, many companies are left with the critical decision of finding alternative solutions. Going beyond the usual feature rundowns and abundant migration guides saturating the web, we performed a focused benchmarking exercise comparing 100ms stack with Twilio. Our mission: unearthing real data that empowers our prospective customers to make informed decisions. 

This post inaugurates our series, evaluating the audio and video quality on 1:1 calls over the web. Before we delve into the finer details of the test setup, let’s take a quick sneak peek at the different types of media servers used on both platforms and how they leverage media quality and enhance end-user experience.

### P2P vs SFU architecture

In the Peer-to-Peer (P2P) model, clients form direct mesh connections to exchange media. While this setup is optimized for one-on-one calls, its scalability diminishes beyond 3-4 clients, and it lacks support for on-cloud recording and post-processing features. For scenarios involving larger group calls, the Selective Forwarding Unit (SFU) comes into play. In this configuration, each client uploads their media once to the server, which then redistributes it to all connected clients. Although SFU optimizes for uplink efficiency, downlink remains resource-intensive, with each client receiving N-1 streams from all publishing clients. To alleviate the demand on download bandwidth, many SFU architectures implement Simulcast. This involves the publishing client sending multiple renditions of the media stream, and the SFU forwards the optimal rendition to each client based on their available bandwidth capacity.

![mesh routing arch](https://storage.googleapis.com/100ms-cms-prod/cms/mesh_routing_arch_9660f03f7b/mesh_routing_arch_9660f03f7b.jpeg?updated_at=2023-12-15T14:25:27.445Z)

![sfu routing arch](https://storage.googleapis.com/100ms-cms-prod/cms/sfu_routing_arch_6f01e46ef8/sfu_routing_arch_6f01e46ef8.png?updated_at=2023-12-15T14:25:28.935Z)

There are a lot of nuances in the SFU while tuning for quality and efficiency. One of the hardest things to get right in an SFU is how to handle varying network conditions. In the real world, the upload and download speeds of a client keep changing. When the network is good, SFU should send media at the highest quality possible. Conversely, during network fluctuations, SFU should respond quickly by reducing the quality, so that video and audio don't freeze. Implementing an effective congestion control system including simulcast support, handling subscribe degradations, and support for temporal scaling makes this a most critical component to benchmark for the high-quality end-user experience. 


### Test set up

In this test, our primary focus is evaluating how Twilio and 100ms SFUs perform against each other under simple constraints. Twilio supports both P2P rooms and Group rooms (SFU-based), while 100ms exclusively supports SFU-based media routing. For this test, we created SFU-based rooms with Simulcast enabled on both Twilio and 100ms platforms. To make the source video exactly the same, we used OBS virtual camera and played a local video file in a loop.

We precisely matched simulcast layer settings, including resolution, frame rate, and bitrate, between Twilio and 100ms for a fair comparison. These settings play a significant role, making their uniformity crucial. The complete set of input parameters for the test is listed in the table below.

|     Parameters     |          Values          |
|:------------------:|:------------------------:|
| SFU                | Enabled                  |
| Simulcast          | Enabled                  |
| Video codec        | vp8                      |
| Audio codec        | opus                     |
| Simulcast layer #1 | 720p / 24 fps / 2500Kbps |
| Simulcast layer #2 | 360p / 24fps / 800Kbps   |
| Simulcast layer #3 | 180p / 24 fps / 100Kbps  |


### Network Bandwidth Shaping Test

In our quest to mimic real-world network fluctuations, we decided to keep things simple rather than getting bogged down with complex tests. We started with a test that only isolated network changes.

1. We focused solely on 1:1 calls over the web.
2. Throttling was applied only to receiver bandwidth, while upload bandwidth remained unlimited.
3. Both delay and packet loss were consistently set to 0.

Even with this stripped-down approach, we can gain some valuable insights into the SFU’s congestion control capabilities. We wrote a Python script that does the traffic shaping by going through a predetermined series of bitrates (refer to the image below). WebRTC exposes a wealth of metrics to make this kind of quality comparison easy. Our script, then, queries WebRTC stats every second, aggregates some data, and prints these values to stdout for later analysis.

![bandwidth in kbps vs time in seconds](https://storage.googleapis.com/100ms-cms-prod/cms/Bandwidth_in_Kbps_vs_Time_in_seconds_375b376b75/Bandwidth_in_Kbps_vs_Time_in_seconds_375b376b75.png?updated_at=2023-12-15T14:25:27.135Z)

As we can see from the chart above, the test starts with a very high bandwidth for the receiver and SFU sends the highest-quality stream available. After 20 seconds, the download bandwidth is throttled to 150Kbps. This tests how quickly the SFU is able to adjust to a sharp fall. Subsequently, a phase of sustained low download speed ensues, marked by some variations. At the 740-second mark, the download bandwidth increases to 1200Kbps and then 1500Kbps. This segment tests how quickly SFU can raise the quality as network congestion eases. The majority of the test is between 150Kbps to 500Kbps, providing insights into the SFU's performance under sustained low download speeds.


### Results

To compare the impact on end-user quality, we analyzed the following key metrics in particular. 

#### [Video Freeze Count](https://www.w3.org/TR/webrtc-stats/#dom-rtcinboundrtpstreamstats-freezecount)

This metric is used to quantify the instances when a video stream appears to freeze or stop during the call. For the end user, these are seen as small stutters in the video.

![number of freezes](https://storage.googleapis.com/100ms-cms-prod/cms/Number_of_freezes_6fe25ca964/Number_of_freezes_6fe25ca964.png?updated_at=2023-12-15T14:25:27.115Z)

#### [Video Freezes duration](https://www.w3.org/TR/webrtc-stats/#dom-rtcinboundrtpstreamstats-totalfreezesduration)

This metric serves as a comprehensive measure of the overall user experience, influenced by how quickly an SFU can adjust to bad network conditions.

![duration of freezes in seconds](https://storage.googleapis.com/100ms-cms-prod/cms/Duration_of_freezes_in_seconds_359d66c357/Duration_of_freezes_in_seconds_359d66c357.png?updated_at=2023-12-15T14:25:27.814Z)

#### [Audio concealed samples](https://www.w3.org/TR/webrtc-stats/#dom-rtcinboundrtpstreamstats-concealedsamples)

Audio sample concealment, if excessive, can be heard as distorted audio and reduces the audio quality.

![audio samples concealed](https://storage.googleapis.com/100ms-cms-prod/cms/Audio_samples_concealed_3857cff244/Audio_samples_concealed_3857cff244.png?updated_at=2023-12-15T14:25:27.104Z)


#### Amount of time spent in each resolution, frame rate combination

More time at higher resolutions and frame rates indicates higher quality video received in the call.

![time spent in each layer](https://storage.googleapis.com/100ms-cms-prod/cms/Time_spent_in_each_resolution_fps_77d26b3477/Time_spent_in_each_resolution_fps_77d26b3477.png?updated_at=2023-12-15T14:25:27.134Z)

Twilio spent the most time in the lowest quality layer (180p12) and slightly better at higher quality (360p12) compared to 100ms. 

### Videos

Video recordings during the initial 30 seconds, when bitrate drops sharply, show the effect on audio/video during sharp network transitions.

**Twilio**
<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/twilio_test_screen_recording_9a30802548/twilio_test_screen_recording_9a30802548.mov?updated_at=2023-12-15T14:27:00.752Z" type="video/mp4" />
</video>

**100ms**
<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/100ms_test_screen_recording_84ec00f3bf/100ms_test_screen_recording_84ec00f3bf.mov?updated_at=2023-12-15T14:25:50.169Z" type="video/mp4" />
</video>


### Conclusion

On analyzing the key metrics from the test results, it becomes apparent that 100ms exhibits a clear advantage over Twilio. With fewer freezes and shorter freeze times, 100ms keeps video more stable and smooth. The audio quality is also noticeably better with 100ms, with lesser packet losses and concealments required. When we talk about the visible picture quality (resolution) and frames per second (how smoothly the video runs), 100ms performs marginally better in extremely low bandwidth conditions. Though Twilio rendered a higher resolution video layer a little longer than 100ms, there is a bigger difference between the two providers at the lowest quality tier. We see that Twilio had longer stretches of very ‘bad’ video - low clarity along with longer freezes whereas 100ms playback suffered from lesser freezes even if the clarity is poor. 

Overall, 100ms performs better than Twilio in very bad network conditions and is comparable when network conditions are stable.


### Resources
[Github repo for the quality benchmarking test](https://github.com/100mslive/webrtc-benchmark/blob/main/README.md)

thumbnail_twilio.png

twilio.png

Comparing Call Quality: 100ms vs. Twilio in 1:1 Web Calls

100ms Team

Zoom is a go-to video conferencing tool for a lot of folks. It caters to a diverse range of use cases with its impressive array of features for all sorts of virtual meetings. Its widespread adoption is a testament to its reliability in facilitating seamless virtual communication, making it a preferred choice for numerous individuals and businesses alike.

However, even with its extensive capabilities, it's crucial to acknowledge that Zoom is not without its limitations. The need for custom code handling to make sure it works, adds a layer of complexity to the integration process. Furthermore, Zoom's reliance on a proprietary implementation of the H.264 video codec and its lack of native support for WebRTC can pose hurdles for those seeking more standardized and open solutions.

In this article, we've compared some of the core areas where developers may encounter challenges when integrating Zoom SDK. We will briefly touch upon how 100ms SDK which solves for these challenges so developers spend less time writing low level handling and build high quality applications. 

## A Brief Summary: Comparing Zoom SDK and 100ms SDK

| Feature | Zoom SDK | 100ms SDK |
|-------| --------| ------|
| SDK size | 75MB | 4MB without compressing, [more details](https://bundlephobia.com/package/@100mslive/hms-video-store@0.10.27) |
| Max tiles with audio/video on in a single page | 9 tiles | 49 tiles |
| Automatic bandwidth management| Custom code required to handle audio and video track degrade. | Automatic bandwidth management with Simulcast control with 100ms dashboard |
| Max Quality | 720p | 1080p |
| Live streaming | Not available | One SDK for both conferencing and livestreaming |
| Cloud Recording | Limited customisations available | Flexible recording APIs to compose tracks
| State Management | Lacks dynamic state management | Real-time component specific state management|
|Webhooks | Limited to cloud recording scenarios | Real-time notification updates sent to your server |
| Analytics | Aggregated analytics limited to a session | [Rest APIs](https://www.100ms.live/docs/server-side/v2/api-reference/analytics/overview) to build insights on peer and session level|

## Zooming into SDK feature gaps

Zoom's SDK poses challenges for developers, requiring intricate code handling. From complex video rendering and specifying coordinates to additional code for features and limitations on 720p video rendering, Zoom introduces complexities. Audio issues on Firefox and Safari, along with managing states, add to the developer's workload. 

### Rendering video tiles is complex application side code

Zoom allows rendering video streams on a canvas based on changes in the peer video state.  Key challenges that developers come across:

- Rendering video requires coordinates for rendering on the canvas based on where you want it to appear. The developer ends up computing dynamic coordinates every time the number of tiles changes/for different screen sizes.
- Rendering on canvas is difficult. With absence of `<audio>` and `<video>` elements, it robs the developer of the power of HTML that can easily create dynamic tile layouts.
- Developers need to manually specify the desired video quality for optimal display. The best practice is to reduce quality when tile sizes reduce, and vice-versa.

### Specify coordinates on canvas to render video tiles

With Zoom’s SDK, developers must handle intricate handling on application side such as canvas rendering, managing participants( who joined, who left, active speaker, where to place them - here’s [the math for canvas to render multiple videos](https://developers.zoom.us/docs/video-sdk/web/video/#render-multiple-participant-videos) with Zoom, and determining the placement of elements. 

Unlike Zoom, 100ms SDK automatically attaches the source to video element wherever it is present on the UI. The element can be customised with CSS.

### Additional code for rendering Gallery View, Virtual Background and 720p videos

With Zoom SDK, you have to use [SharedArrayBuffers](https://developers.zoom.us/blog/sharedarraybuffer-overview-and-implementation/) which adds another layer of complexity. Without SharedArrayBuffers, features like Gallery View, Virtual Background, and 720p video are unavailable. See one of the developer’s struggling https://devforum.zoom.us/t/struggling-with-sharedbufferarray-and-cors/87881

Setting up ShareArrayBuffers for rendering multiple videos requires one of the following to be setup:

1. [Cross-Origin Isolation](https://developer.chrome.com/blog/enabling-shared-array-buffer/#cross-origin-isolation)
2. [Credentialless Headers](https://developer.chrome.com/blog/coep-credentialless-origin-trial)
3. [Using Chrome's Origin Trials](https://developer.chrome.com/blog/enabling-shared-array-buffer/#origin-trial)

### Maximum one 720p video

Zoom SDK restricts rendering only one 720p video on a webpage at a time, making grid-style rendering with higher quality challenging ([Zoom Video SDK Documentation](https://developers.zoom.us/docs/video-sdk/web/video/#render-multiple-participant-videos)).

### Audio Challenges on Firefox and Safari Desktop

Getting audio started on Firefox and Safari desktops requires additional logic, as outlined in the [Zoom Video SDK documentation](https://developers.zoom.us/docs/video-sdk/web/audio/#start-audio). This extra step may pose challenges for developers seeking seamless audio functionality across different browsers.

### Managing states across your application

Managing state in a web app can be tricky, but it's crucial for things like keeping track of who's in a room and recording user actions to be used across your app.

With Zoom, you have to maintain who is present in the room, which user has performed which action etc.

- The challenge lies in avoiding inconsistent states, depending on how the state is handled.
- Managing this across various pages and components can be challenging.

The provided [sample code](https://github.com/zoom/videosdk-web-sample/blob/master/purejs-demo/src/js/meeting/session/simple-state.js) by Zoom for maintaining the state in a Zoom application is rather basic and has certain limitations:

- It represents a minimal state that lacks the comprehensive information necessary for rendering a true video conferencing UI e.g who’s screensharing, is there a screenshare present etc.
- It lacks dynamic updates, leaving it up to the developers to listen to the correct events and update states. This often causes race conditions (eg. a user removed called without the number of participants having been updated), often resulting in common user errors - “I’ve joined, but others can’t see me”
- The necessity to pass around elements such as **`zoomClient`**, **`mediaStream`**, and audio/video tracks wherever they are needed introduces complexity, potentially hindering the maintenance of an efficient codebase.

100ms uses HMSReactiveStore as a store for managing reactive state in a video application. It also sets up the store to notify subscribers about changes immediately upon subscription, exports the store's actions and store, making them available for other parts of the application. This store basically aims to solve the limitations of Zoom state management with:

- **Real-time Updates - It** ensures you always have upto date information across your app, and don’t have to instrument event handlers or deal with race conditions around this management
- **Component-Specific State -** Attach a particular piece of state to a component, and it automatically reflects the updated state whenever it changes.
- **Managed Updates -** Only predefined actions can update the store, ensuring internal management and handling potential race conditions (like consecutive audio/video toggles) with precision.

## Finally, making a choice

While Zoom’s client is great for video conferencing, in our opinion, Zoom Video SDK leaves a lot of the heavy lifting to developers for an ideal user experience. In practice, this adds several cycles of error handling to just get close to production quality. Their non-standard webRTC implementation also adds a ceiling to the user-experience that can be achieved in a browser.

Custom video SDKs like 100ms, come baked in with sensible defaults (state management, video layouts using HTML, dynamic quality changes) - that allows developers to go live with fewer errors, and higher quality user experiences. Read more on about 

## Try 100ms now
We believe in empoworing developers to create rich, real-time applications with simplicity and compatibility abstracting low level code handling. Some of the key features of 100ms Web SDK: 
- **Priorities**: Emphasizes simplicity in code and easy debugging
- **Customization**: Enables customization of video/audio calls, streaming, and collaborative tools
- **API Surface**: Small, intuitive, TypeScript-based API surface minimizes documentation needs
- **Abstraction Layer**: Acts as an abstraction layer, handling real-time application complexities
- **Flux Architecture**: Guides video call breakdown into actions and states for a well-designed codebase
- **hmsActions and hmsStore**: Links actions and states, streamlining code and reducing errors
- **JavaScript Integration**: Seamless integration with React, Angular, VueJS, and Svelte

For more details, do check out [Web SDK mental model.](https://www.100ms.live/docs/javascript/v2/quickstart/mental-model). Want to start building? Head to [JS quickstart guide.](https://www.100ms.live/docs/javascript/v2/quickstart/javascript-quickstart)

thumbnail_100ms-vs-zoom.png

100ms-vs-zoom.png

Comparing Zoom video JS SDK to 100ms

FixHealth is a global platform that provides access to high-quality physiotherapy care. Started in 2020, by Dr. Sheetal Mundhada, the team has built a platform that aims to make pain relief as accessible and convenient as watching a Netflix movie. Physiotherapy sessions occur at the patient’s home, with no wait time, and are administered by experienced, high-quality physiotherapists.

Since its launch, the platform has grown to hundreds of thousands of sessions across thousands of cities in the US, UK, and India.

### Background

FixHealth’s use case for video was straightforward: add a video calling feature in their web app for video consultations between patients and their physiotherapists.

Initially, the team provided 1-on-1 physiotherapy sessions online via Zoom. However, their vision was always to incorporate technology to aid the physiotherapist. By utilizing modern posture detection machine learning models, they could measure outcomes across sessions and provide real-time feedback to patients.

### Initial Implementation
Raunak Khandelwal, co-founder, initially selected Twilio to build an integrated live video 1-on-1 call with TensorFlow models running on the physiotherapist’s browser, detecting posture in real-time.

However, they quickly encountered two issues:

- Twilio didn’t offer a natural plugin interface to access video frames, leading to double rendering of video – once on a hidden canvas for ML pipelines and once for actual display.
- Twilio left much of the audio-video optimization, including handling disconnections and setting optimal initial settings, to the developer, resulting in a noticeable drop in video quality.

After two months of experimentation, physiotherapists expressed dissatisfaction with the sluggish performance of the systems, attributed to the inefficiencies in the machine learning models, and raised concerns about subpar audio-video quality. The video quality would have frequent drops, even under favourable internet conditions.

![](https://storage.googleapis.com/100ms-cms-qa/cms/Fix_Health_3_a8f55d666c/Fix_Health_3_a8f55d666c.png?updated_at=2023-12-14T15:34:54.321Z)

### Switching to 100ms

When 100ms approached them, Raunak was immediately excited by our first-class [plugin interface](https://www.100ms.live/docs/javascript/v2/how-to-guides/extend-capabilities/plugins/custom-video-plugins) for running ML models.

Lead engineer Ashhar Akhlaque started to build a proof of concept (POC) and integrated their ML pipelines, going live in just one week. The improvements were immediate:

- 100ms’ [template](https://www.100ms.live/docs/get-started/v2/get-started/concepts/templates-and-roles) configuration and built-in degradation handling significantly enhanced video quality.
- 100ms’ [first-class plugin](https://www.100ms.live/docs/javascript/v2/how-to-guides/extend-capabilities/plugins/custom-video-plugins) interface allowed the entire app to run smoothly without slowing down the physiotherapists’ systems.

Over the next two months, they transitioned their customers from Zoom to 100ms, with real-time posture-detection models. 

Their platform now uses ML to aid in root-problem detection, provide real-time feedback to the patient and physiotherapist on posture correctness, and also measure improvement across sessions.

![](https://storage.googleapis.com/100ms-cms-qa/cms/Fix_Health_4_3d488989c9/Fix_Health_4_3d488989c9.png?updated_at=2023-12-14T15:34:54.422Z)

![](https://storage.googleapis.com/100ms-cms-prod/cms/Fix_95b2d702d3/Fix_95b2d702d3.mov?updated_at=2023-12-14T15:45:50.155Z)

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/Fix_95b2d702d3/Fix_95b2d702d3.mov?updated_at=2023-12-14T15:45:50.155Z" type="video/mp4" />
</video>


thumbnail_FixHealth-2.png

FixHealth-2.png

How FixHealth is leveraging AI with Live Video Stack


#### The Release We've All Been Waiting For!🥳

Hello there!👋🏻 We're absolutely thrilled to share some exciting news about the latest 100ms Prebuilt update. We've been listening closely to feedback from our customers; both developers and PMs and our [Discord community](https://discord.com/channels/843749923060711464/1159469649260462131), and we've got some special updates in store for you.

We're rolling out a set of features that will make your life as a developer much easier when it comes to building audio and video applications. With 100ms Prebuilt, your time to ship decreases drastically; more like a plug and play video widget. These enhancements strike the perfect balance between customization and ease of use, so you can ship amazing things with minimal code.

We're pretty confident that you're going to love what we've cooked up. So, let's dive into the key updates and see how we're taking things to a whole new level!

1. **Ready-to-Ship UI** - We've revamped room controls and designed beautiful video layouts for various use cases such as 1:1 calls, group calls, and livestreaming. These improvements come with well-thought-out default settings, which can be customised no-code with 100ms dashboard.
2. **Embedding Prebuilt Component *with one line of code*** - We understand the importance of accessibility, so we've introduced a video widget that can seamlessly integrate as a native component called `HMSPrebuilt` into your applications. This component complements the existing Prebuilt links and opens up new possibilities for integration. You can find a step-by-step guide in our documentation, or check out this quick blog read that uses a telehealth consultation example.
3. **Customization and Extensibility** -Customization has always been a top priority for developers and PMs. We observed a pattern in the need for simplifying, increasing accessibility, and enhancing configurability in customizations; this implied that a prebuilt solution cannot be a one-size-fits-all approach..


## What's new in 100ms Prebuilt

In this article, we will discuss the updated Prebuilt UX and how you can customize it. We highly recommend signing up or logging in to the [100ms dashboard](https://dashboard.100ms.live/login) and following along with this article to try out 100ms Prebuilt for yourself.


### Prebuilt Customiser

Prebuilt now offers more configurability with the new Prebuilt Customiser on the 100ms dashboard. This tool allows you to customize various aspects of Prebuilt and provides a side-by-side preview so you can visualize the changes you're making. The preview uses `HMSPrebuilt` component for showing the side-by-side preview. Broadly, there are two customisation levels available you can do:

1. Change **Appearance -** Customize Prebuilt's default colors to align with your brand's theme and update your brand logo. If you like, you can switch to a light theme, as Prebuilt defaults to a dark theme. Please note that the any changes to Appearance section apply to all roles on Prebuilt.
2. **Screens and Components -** This feature provides you with deeper control over how Prebuilt behaves for different roles. Simply select a role and Prebuilt screen, then configure specific features for that role. 

For instance, you can set up features for different roles in your template.
            
| Feature      | Host | Guest |  Config |
| ----------- | ----------- | ----------- | ------|
| Emoji Reactions      |   -     | ✅ | Only Guests role can send emojis; Host can't
| Chat   |   ✅      | ✅ | Both Host and Guest roles can send chat
| Be Right Back   | -        | - | No role can set themselves on BRB
| Participant List   | ✅        | -| Only Host can see room participants, while Guest cannot



#### Try Prebuilt customiser 

1. Go to a Webinar Template on 100ms Dashboard and choose “Customise Prebuilt”
2. Update Appearance and add custom themes to your template’s Prebuilt UI
3. Choose any role and select a screen to customise. Watch preview on the side to see your changes
4. We’ve also shipped Prebuilt native apps that uses Prebuilt Component `HMSPrebuilt` which honours customisations on the dashboard. You can play around with customisation and test the UI on Android, iOS, Flutter, React and Web.


Note : We've a detailed blog coming up Prebuilt Customisation, subscribe to the blog updates or join our Prebuilt Discord community for latest updates.

### Preview Screen
This is the first of the three screens on Prebuilt. This is where your users land before they join a 100ms hosted room. We've added some sleek updates to make joining more seamless:
1. Customise Title and Subtitle on preview when your users enter the room - Delight your users with a custom welcome text for each role in your room.
2. Users can quickly access microphone and video settings before joining the room

### New Video Tile Layouts
1. **Two ways to do a 1:1 call -** Whether it's a 2-peer call or a large room, participants should prioritize watching remote peer tiles over their own videos. That's why we have introduced **Inset Tile Mode as the default mode for local peer tiles.** This mode allows for FaceTime-like calls on both web and mobile platforms. However, you have the option to disable this mode and switch to equal prominence mode, where the tile size is the same for all peers.
It's important to note that a 1:1 call can scale from a small group call to a large group call, supporting up to 20,000 participants.

2. **Enhanced Grid View** - 100ms Prebuilt dynamically arranges tiles that optimises for maximum tile coverage area; the layouts are now more responsive and adaptive with Chat and Participant list.
3. **Prominent tiles for roles** - For a 1:many use case, you can now assign a role to be the most prominent role in the room, while other roles can be assigned secondary tile position. 
3. **Pinning and Spotlighting** - Pinning or spotlighting yourself or someone pins the video tile like below:

| Pinning      | Spotlighting 
| ----------- | ----------- 
| When you pin yourself or someone, it only applies only to local view.      | When you spotlight yourself or someone, it spotlights the tile for everyone in the room
| It's available for everyone in the room. |  Enable Spotlight feature for specific roles from Prebuilt customiser |

### New Chat and Participant List Experience

To improve accessibility, users can easily switch between the Chat and participant list sections from the footer. This feature enables seamless navigation and efficient management of both Chat and participant interactions.

#### Chat

We now offer two types of Chat components. Broadcasters who go live from mobile can now use overlay/transparent chat, allowing for uninterrupted viewing and engaging with viewers through chat.

#### Participant List

1. The roles of participants in the room are now conveniently grouped together in the Participant list.
2. Additionally, any instances of hand raises in the room will be displayed as a distinct container within the participant list.

### Scaled for large rooms
With our recent infra updates, Prebuilt gracefully scales from a 1:1 to a large video call supporting upto 20,000 participants in a session. [to add more content here]


### Bring On Stage
In a webinar with a Broadcaster and multiple Viewers (audio/video permissions); Viewers can request to talk to broadcaster real-time by raising their hands. Broadcaster can approve/decline viewers requests to join the broadcaster.

## Deprecated from 100ms Prebuilt UI

We’ve decided to temporarily deprecate some features from 100ms Prebuilt, either because they were not usable, not user-friendly, or simply not used much by our customers. In order to maintain the best features for our customers, we have chosen to deprecate these features. We plan to re-introduce these features with an improved experience as part of our roadmap.

1. Audio Playlist
2. Video Playlist
3. Audio Sharing
4. Embed and Share URL

## This is not the end

This release marks the start of our journey to make video creation as low-code as possible. No-code customization is now a reality with Prebuilt. We have a lot of exciting updates planned for the future, here's a peak into the list of upcoming features on Prebuilt:

- Polls and Quizzes, which allow users to engage with their audience and gather feedback.
- Whiteboard, a collaborative tool that enables users to brainstorm and visualize ideas.
- Disabling a room, giving users the ability to temporarily pause or close a room without deleting it.
- Live captions, providing real-time transcriptions for improved accessibility.
- Advanced webinars controls like blocking users from rejoining, automatically ending the room when alone
- And much more, as we continue to innovate and expand our feature set to enhance your experience on Prebuilt.

We encourage you to share your feedback and report any issues you come across on our [Prebuilt Discord channel](https://discord.com/channels/843749923060711464/1145960444230647808). Your input is extremely valuable as we continue to improve 100ms Prebuilt and tailor it to your needs. By the way, our team loves reading the use cases you build with us❤️

thumbnail_Prebuilt.png

Prebuilt.png

The all new 100ms Prebuilt 

thumbnail_Prebuilt Flutter - 3.png

Prebuilt Flutter - 3.png

Test

Video calls have changed the way we work, learn, and even see a doctor. But behind the scenes, making video calls work well is actually pretty hard. It's like trying to make sure the picture and sound are perfect for everyone, no matter how fast their internet is or what device they're using. And if that's not tricky enough, adding things like chat and synchronized actions feels like too much of handling when you just want to ship faster. If you've ever felt the struggle, don't worry – a lot of us do. 

That's why, ***Drum rolls, please!*** 🥁 At 100ms, we've developed an SDK with a fully equipped, configurable, and customizable Prebuilt UI support. It's presented as the 100ms Prebuilt Flutter component from the `HMSRoomKit` package. With just a few simple steps, you can add a video widget by just adding necessary dependencies, and presto! We've handled everything, from initiating calls to crafting the user interface – making integration a breeze. Now, isn't that exciting?! 🎉

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/really_838347e300/really_838347e300.gif) 

In this article on 100ms Flutter room kit integration assumes a familiarity with **Dart** and **Flutter**.


----------



## Knowing 100ms Prebuilt
Understanding what you're adding to your app is crucial. Here's a quick rundown of key points about 100ms Prebuilt before you dive into integrating it:

1. **Designed with good defaults** - 100ms Prebuilt can be tailored for various scenarios like one-on-one sessions, group calls, webinars, and livestreaming. It comes with predefined roles and permissions, offering a ready-to-go prebuilt UI designed specifically for each role in your template. 
2. **Plug and play video as a native component** - Embed live video experiences with no-code that can be seamlessly embedded to your apps. In this article, you will learn how easy it is to add video calling in your applications using our Flutter Prebuilt package `HMSRoomKit`.
2. **Manage and customize with no-code** - 100ms Prebuilt is an abstraction on top of a template configuration you can configure on 100ms dashboard, while you can customize Prebuilt Appearance, behavior for each role with Prebuilt customizer.
2. **Customize with code** - Just like many developers who appreciated 100ms Prebuilt and wished to fine-tune it to fit their precise needs, we've open sourced the Flutter Prebuilt component so you can extend upon Prebuilt's default UI to match exactly like you want.

Currently, 100ms Prebuilt offers a single powerful widget for end-to-end meeting flow, going forward we will be exposing individual components from the package, so don’t forget to Watch and Star the [Github](https://github.com/100mslive/100ms-flutter) repo to stay updated.



----------


## Embedding 100ms Prebuilt Flutter component 

We will create a simple video calling and streaming application using `HMSRoomKit` package.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/step_by_step_process_99db93656f/step_by_step_process_99db93656f.gif) 

### Prerequisites
1. **Set up a template** - A [template](https://www.100ms.live/docs/get-started/v2/get-started/concepts/templates-and-roles) governs what features your Prebuilt has and how its Prebuilt UI functions. Head over to [100ms dashboard](https://dashboard.100ms.live/login) to create a template.

2. **Create a room** - A default room gets created for any template created using the 100ms dashboard. Create more rooms under the same template programmatically using the [create room API](https://www.100ms.live/docs/server-side/v2/api-reference/Rooms/create-via-api).

3. **Create and get room codes** - [Room code](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/overview) represents a unique room_id and role combination; room codes can help create room links or power authentication when joining a room. Get room codes for each role
    - from Dashboard - Copy role specific room code from Join Room section for a role in a room
    - via API - [Use Create Room Codes API](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api) to create room codes and get room codes using [retrieve room codes API](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/get-room-code-api).

### Import Room Kit
Add the `hms_room_kit` package under dependencies in your Flutter project’s `puspec.yaml` file.
    
   ```yaml
    dependencies:
      flutter:
        sdk: flutter
    
      # Replace `1.0,8` with whatever is the latest version
      hms_room_kit: ^1.0.8
   ```
    
  To know more about the package, check out the Flutter repo [here.](https://github.com/100mslive/100ms-flutter)
    
### Embed `HMSPrebuilt`
Now, simply place the `HMSPrebuilt` widget wherever you need it. For instance, in this example, a button labeled "Join" triggers navigation to the meeting UI upon clicking. You’ll notice that joining a room requires a room code which is unique to each role in your template.
    
   ```dart
    Scaffold(
          body: Center(
            child: ElevatedButton(
              style: ButtonStyle(
                  shape: MaterialStateProperty.all<RoundedRectangleBorder>(
                      RoundedRectangleBorder(
                borderRadius: BorderRadius.circular(8.0),
              ))),
              onPressed: () async => {
                await Navigator.push(
                  context,
                  MaterialPageRoute(
                    builder: (context) => const HMSPrebuilt(roomCode: "Enter your room code here")
                  ),
                ),
              },
              child: const Padding(
                padding: EdgeInsets.symmetric(vertical: 20, horizontal: 20),
                child: Text(
                  'Join',
                  style: TextStyle(fontSize: 20),
                ),
              ),
            ),
          ),
        );
   ```
- For Android, in your `android/app/build.gradle` file (inside "defaultConfig"), update the `minSdkVersion` to 21.

```dart
    ...
    defaultConfig {
        ...
        minSdkVersion 21
        ...
    }
    ...
   ```

- For iOS, in `ios/Podfile` add the target platform version:

    ```dart
    platform :ios, '12.0'
    ```

And we’re done! Time to run the application.

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/flutter1_2_038e4f175a/flutter1_2_038e4f175a.mp4?updated_at=2024-01-01T09:46:44.815Z" type="video/mp4" />
</video>

### Adding Screen share support

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Prebuilt_Flutter_2_7d8de84c86/Prebuilt_Flutter_2_7d8de84c86.png) 

#### Screen share support on Android
Add the below code in `android/app/src/main/kotlin/live/hms/flutter/MainActivity.kt` file:
    
   ```dart
    import android.app.Activity
    import android.content.Intent
    import live.hms.hmssdk_flutter.Constants
    
    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)
    
        if (requestCode == Constants.SCREEN_SHARE_INTENT_REQUEST_CODE && resultCode == Activity.RESULT_OK) {
            data?.action = Constants.HMSSDK_RECEIVER
            activity.sendBroadcast(data?.putExtra(Constants.METHOD_CALL, Constants.SCREEN_SHARE_REQUEST))
        }
    }
   ```
 
#### Screenshare support on iOS
For screen sharing in iOS please follow [this doc](https://www.100ms.live/docs/flutter/v2/how-to-guides/set-up-video-conferencing/screen-share#ios-setup). 
Pass the `iOSScreenshareConfig` in the `HMSPrebuiltOptions` parameter of the `HMSPrebuilt` widget to enble screen share in your app.
   ```dart
   // Pass the correct App Group & Preferred Extension parameters in HMSIOSScreenshareConfig class.
 
    HMSPrebuilt(
        roomCode: meetingLinkController.text.trim(),
        options: HMSPrebuiltOptions(
            iOSScreenshareConfig:  HMSIOSScreenshareConfig(
        appGroup: "appGroup", // App Group value linked to your Apple Developer Account
        preferredExtension: "preferredExtension" // Name of the Broadcast Upload Extension Target created in Xcode
    )));
   ```
    
### Adding PIP support
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Prebuilt_Flutter_3_b2e36c957f/Prebuilt_Flutter_3_b2e36c957f.png) 
    
#### Adding PIP support on Android
- To add PIP support in your app manifest file("android/app/src/main/AndroidManifest.xml") add:

 ```dart      
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
     <activity
       ....
       android:supportsPictureInPicture="true"
       android:configChanges="screenSize|smallestScreenSize|screenLayout|orientation"
       ... />
 ```           
- For enabling auto enter PIP(on back gesture), in your `MainActivity.kt` file("android/app/src/main/kotlin/live/hms/flutter/MainActivity.kt") add:
      
```dart
    import android.content.res.Configuration
    import android.os.Build
    import live.hms.hmssdk_flutter.methods.HMSPipAction
    import live.hms.hmssdk_flutter.Constants

    override fun onUserLeaveHint() {
      super.onUserLeaveHint()
      // This should only work for android version above 8 since PIP is only supported after
      // android 8 and will not be called after android 12 since it automatically gets handled by android.
      if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O && Build.VERSION.SDK_INT < Build.VERSION_CODES.S) {
        HMSPipAction.autoEnterPipMode(this)
      }
    }
  ```
        
#### Adding PIP support On iOS
 There are some minimum requirements for running PIP in iOS:

- The minimum iOS version required to support PiP is iOS 15
- Your app should have **[com.apple.developer.avfoundation.multitasking-camera-access](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_avfoundation_multitasking-camera-access)** Entitlement to use PiP Mode.
- Your app needs to run on iOS 13.5 or later to use the entitlement. Without the entitlement, the system disables the camera access for your app. When your app enters PIP mode, it needs this entitlement to continue using the camera.
- After you receive permission from Apple, add the Entitlement to your app by opening the Entitlements file in Xcode. Add the key and set the corresponding value to `YES`.

That's all there is to it! These steps will help you create a fully functional video-calling application in Flutter.

Run your application and test it out!

## Customizing your Prebuilt Flutter app

Now that you have the 100ms Prebuilt component embedded in your Flutter app, you might want to tweak it further to appeal it more to your use case. 100ms Prebuilt allows complete no code customization with 100ms dashboard using;
- Templates - Customize default roles and set up features for your Prebuilt
- Prebuilt Customizer - Customizer Appearance, screens and components functionality

Here's a quick guide on how you can [customize Prebuilt](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/overview#customize-prebuilt). 

<video loop="true" autoplay="autoPlay" controls="false" id="vid" muted>
    <source src="https://storage.googleapis.com/100ms-cms-qa/cms/customise_a35a93e25a/customise_a35a93e25a.mp4?updated_at=2024-01-02T09:59:44.716Z" type="video/mp4" />
</video>


Our Prebuilt roadmap is loaded with awesome plans. If you're eager to join this journey and be a part of the action, contributing your expertise, we'd love to have you on board! 🌟, do consider joining our [#Prebuilt discord channel](https://discord.gg/jD94Fp74Ea). Adiós👋

thumbnail_Prebuilt Flutter - 1.png

Prebuilt Flutter - 1.png

Video Calling Application in Flutter using 100ms Prebuilt

Building a video-conferencing or streaming app from scratch is no small feat. From aligning audio and video streams to designing user-friendly interfaces, it can be a challenging task with many factors to consider. That's where **`HMSRoomKit`** comes in - it is designed to make this process easier for you by providing prebuilt conferencing and live-streaming components that are ready to use.

## Why HMSRoomKit?

At its core, `HMSRoomKit` is inspired by the simplicity and clarity of SwiftUI's declarative approach. By aligning itself with these principles, it transforms the often daunting task of creating video-conferencing and live-streaming interfaces into an intuitive experience. For a glimpse of how the code has become more intuitive, let’s see how you’ll build a simple conferencing screen with RoomKit. In the example below, we construct a conference screen with tiles for call participants and controls for the microphone, camera, and exit functions:

```swift
import HMSRoomKit
import HMSRoomModels

struct MeetingView: View {
    
    @ObservedObject var conferenceRoom = HMSRoomModel(roomCode: "qfw-mit-seb")
    
    var body: some View {
        
        VStack {

            HMSPeerLayout()

            HStack(spacing: 30) {
                Spacer()
                Image(systemName: conferenceRoom.isMicMute ? "mic.slash" : "mic")
                    .onTapGesture { conferenceRoom.toggleMic() }
                Image(systemName: conferenceRoom.isCameraMute ? "video.slash" : "video")
                    .onTapGesture { conferenceRoom.toggleCamera() }
                Image(systemName: "phone.down.fill")
                    .onTapGesture {
                        Task { try await conferenceRoom.leave() }
                    }
                Spacer()
            }
            .padding()
        }
        .environmentObject(conferenceRoom)
        .environmentObject(HMSUITheme())
    }
}
```

With the provided code, you can achieve a fully-functional conference screen as shown below:
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Prebuilt_i_OS_3_b3046bd117/Prebuilt_i_OS_3_b3046bd117.png) 

### A Glimpse into how HMSRoomKit’s Prebuilt View simplifies video app development

At its core, `HMSRoomKit` offers a suite of **prebuilt components** designed to streamline the creation of video-conferencing or live-streaming apps. The best manifestation of this simplicity shines through the `HMSPrebuiltView` offered by the SDK. This is great for developers keen on speedy implementation. 

> `HMSPrebuiltView` is not just any component - it encapsulates the complete video-conferencing and live-streaming experience within a singular SwiftUI view. Imagine having a video conference interface up and running in mere minutes. With `HMSPrebuiltView`, that's entirely possible.

```swift
import HMSRoomKit

struct ContentView: View {
    var body: some View {
        HMSPrebuiltView(roomCode: /*pass room code as string here*/)
    }
}
```

And it's done! `HMSPrebuiltView` is not only easy to use but also empowering. For many applications, this might be the only component you'll need. Moreover, it's supported by a comprehensive, no-code web interface. This allows you to customize the look and feel of your meeting application according to your brand or preference, without needing to delve into complex coding.

<video loop="true" controls="controls" id="vid">
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/HMS_Prebuilt_Mobile_View_1_810aaf55dd/HMS_Prebuilt_Mobile_View_1_810aaf55dd.mp4?updated_at=2023-09-29T07:30:42.124Z" type="video/mp4" />
</video>

</br>

You can use `HMSPrebuiltView` right away with **1.0.0** released version of our **[open source SDK](https://github.com/100mslive/100ms-roomkit-ios)**.  To help you integrate RoomKit into your apps, we have mentioned all the steps you need in the readme file.

## Exploring HMSRoomKit: Using SwiftUI in Real-World Apps

At 100ms, we have embraced the future by using SwiftUI to power our iOS RoomKit SDK. This is a significant step forward, as not many companies are currently using SwiftUI in production. Netflix is one of the few companies that have successfully implemented SwiftUI in their production environment. Before we delve into RoomKit, let's first understand what SwiftUI is.

### SwiftUI: The Shift from Programmatic to Declarative

Let's understand the fundamental shift in UI development - from programmatic to declarative. We can understand the concepts of programmatic (imperative) and declarative programming by examining how we would describe placing a glass of water on a table using each approach.

1. **Imperative (Programmatic) Style:** You explain every step of how to put the glass on the table.

    ``` swift
        let table = Table()
        let glassOfWater = GlassOfWater()
        table.pickUp(glassOfWater)
        table.moveAboveSpot(desiredSpot)
        table.lowerGlass(glassOfWater)
        table.releaseGrip()
      ```

2. **Declarative Style (SwiftUI):** You simply state that you want the glass on the table, and the framework figures out how to achieve it.

      ```swift
        struct TableView: View {
            var body: some View {
                Table {
                    GlassOfWater()
                }
            }
    }
    ```

##### Pros and Cons

The advantage of **Imperative** approach is explicit control meaning developers have granular control over the flow of the program.

The advantages of **Declarative** approach include less boilerplate code, potentially fewer bugs, and easier readability. In real-world coding, especially in UI development with SwiftUI, this distinction means you focus more on defining the desired state of your UI, and less on the sequence of operations to achieve it.

As you can see, the pro for one approach is a con for the other.

### Harnessing the Power of SwiftUI in HMSRoomKit

At its core, `HMSRoomKit` offers a suite of **prebuilt components** designed to streamline the creation of video-conferencing apps. But what truly sets it apart is its deep integration with SwiftUI's declarative approach. Rather than tediously assembling UI pieces and manually updating them with data changes, developers can now rely on reactive models and environment objects to automatically drive the UI. `HMSRoomKit` is not just a collection of components; it's a testament to the power of declarative design. With prebuilt components and deep integration with SwiftUI, it promises a seamless experience for developers.

Just as SwiftUI has simplified general UI development, `HMSRoomKit` looks poised to simplify the complexities involved in crafting video conferencing interfaces. For instance, consider the following simplified example of a virtual meeting view:

```swift
import HMSRoomKit

struct MeetingView: View {
    
    var body: some View {
        
        VStack {
            
            HMSPeerLayoutView()
            
            HStack {
                MicAndCameraToggleView()
            }
        }
    }
}
```

Here, `HMSPeerLayoutView` and `MicAndCameraToggleView` effortlessly lay out the necessary components for a conference room on your screen. But, to truly imbue these components with life and functionality, we have to tie them to a real conference room. Let’s look at how to achieve that.

```swift
import HMSRoomKit
import HMSRoomModels

struct MeetingView: View {
    
    let conferenceRoom = HMSRoomModel(roomCode: "qdf-mik-seb")
    
    var body: some View {
        VStack {
            
            // HMSPeerLayoutView utilizes conferenceRoom present in the environment to drive it's UI
            HMSPeerLayoutView()
            
            HStack {
                // MicAndCameraToggleView shows how you can write your own views that react to conferenceRoom present in the environment
                MicAndCameraToggleView()
            }
        }
        .environmentObject(conferenceRoom)
    }
}


// Definition of MicAndCameraToggleView
struct MicAndCameraToggleView: View {
    
    // conferenceRoom can be accessed in your own view like this
    @EnvironmentObject var conferenceRoom: HMSRoomModel
    
    var body: some View {
        
        Image(systemName: conferenceRoom.isMicMute ? "mic.slash" : "mic")
            .onTapGesture { conferenceRoom.toggleMic() }
        Image(systemName: conferenceRoom.isCameraMute ? "video.slash" : "video")
            .onTapGesture { conferenceRoom.toggleCamera() }
    }
}
```

Here we create an instance of `conferenceRoom` and inject it into the environment. The `conferenceRoom` object is now accessible to every view in the VStack. By doing this, views like `HMSPeerLayoutView` and others know what context they are running under and can adapt their behaviour accordingly. For instance, as a participant joins the conference, they automatically appear in the `HMSPeerLayoutView`. Similarly, `MicAndCameraToggleView` toggles the mic/camera of local user in the `conferenceRoom`.

The following diagram shows **anatomy** of the above `MeetingView` where `HMSPeerLayoutView` is an `HMSRoomKit` view while `MicAndCameraToggleView` is a custom view written by the client. Notice how both of these views can access `conferenceRoom` (`HMSRoomModel`) object from the SwiftUI environment and react to the updates.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Prebuilt_i_OS_2_290620feb2/Prebuilt_i_OS_2_290620feb2.png) 
### **Observable Models: Breathing Life into Real-time Interactions**

Delve into the essence of `HMSRoomKit`, and you'll notice it's not merely about data structures. Models like `HMSRoomModel` and `HMSPeerModel` are dynamic entities that vibrate with changes in real time. Thanks to the `@Published` property wrapper, they automatically notify SwiftUI views about any alterations, establishing a seamless flow of data.

For instance, consider this representation:

```swift
public class HMSPeerModel: ObservableObject {
    ...
    @Published public var isVideoDegraded: Bool
    ...
}
```

In the context above, the `HMSPeerModel` isn’t merely acting as a placeholder for data. It's an active observer, alerting the system of any pertinent changes.

Delving further into the dynamics of these models, one can admire their reactive essence. Take a look at `isVideoDegraded` property of `HMSPeerModel`:

```swift
@Published public var isVideoDegraded: Bool

```

Imagine a situation where a participant's video quality diminishes. The `isVideoDegraded` property in `HMSPeerModel` instantly responds. Instead of entangling your code with intricate delegate methods or callbacks, SwiftUI views organically respond to this change. The outcome is not just streamlined code, but also an enriched user experience that effortlessly corresponds to real-world changes.

### SwiftUI Environments and Environment Objects: The Glue of Consistency

One of the ingenious integrations in HMSRoomKit is its adept utilization of SwiftUI’s environment objects. These global resources, be it `HMSRoomModel` or `HMSUITheme`, become readily accessible to numerous views, underpinning consistent data and behavior throughout.

Consider the scenario where you adjust the UI theme or modify certain room details. With environment objects in place, these transitions effortlessly permeate across the entire app, without the usual chaos of data passing or reliance on singletons.

Think of environment objects in SwiftUI as reservoirs of shared information that underpin coherence across the interface. 

For instance, consider the following example. Here we want to show all peers in a room. Notice how `HMSPeerLayout` view is shown in an environment containing `conferenceRoom` it should use and UI theme it should adapt.

```swift
let uiTheme = HMSUITheme()
let conferenceRoom = HMSRoomModel(roomCode: "qfw-mit-seb")

var body: some View {
    VStack {
        HMSPeerLayout()
    }
    .environmentObject(uiTheme)
    .environmentObject(conferenceRoom)
}
```

Here, any modifications to the UI theme – be it colors, typography, or branded insignia – manifest uniformly across all views, encapsulating a cohesive user interface. Similarly, any **changes in the `conferenceRoom` will be propagated to every view** in the VStack, so if a new peer joins the room they will be shown in the peer layout grid.


## Simplicity & Customizability: The HMSRoomKit Experience

`HMSRoomKit`'s design philosophy revolves around two core tenets: simplicity and customizability. Whether you're a developer looking to quickly set up a video-conferencing feature or one aiming for a bespoke conferencing experience tailored to specific requirements, RoomKit has got you covered.

### Zero to Conference/Live-streaming in Seconds - Released

For developers aiming for rapid deployment of a video-conferencing feature, RoomKit offers the `HMSPrebuiltView`. This component encapsulates the entire** video-conferencing and live-streaming experience** in a single view.

```swift
import HMSRoomKit

struct ContentView: View {
    var body: some View {
        HMSPrebuiltView(roomCode: /*pass room code as string here*/)
    }
}
```

With just a few lines of code, you have a **fully functional** video-conferencing or live-streaming setup. No hassle, no complex configurations.

The following video shows the UI of the app using `HMSPrebuiltView`:

<video loop="true" controls="controls" id="vid">
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/HMS_Prebuilt_Mobile_Viewer_Broadcast_2_b7e04003a0/HMS_Prebuilt_Mobile_Viewer_Broadcast_2_b7e04003a0.mp4?updated_at=2023-09-29T07:32:20.221Z" type="video/mp4" />
</video>

You can use `HMSPrebuiltView` right away with **1.0.0** released version of our **[open source SDK.](https://github.com/100mslive/100ms-roomkit-ios)** 

The next two examples will be part of future releases, but if you are keen, you can try them out with the main branch of [100ms-roomkit-ios](https://github.com/100mslive/100ms-roomkit-ios)

### Tailored Conference Experiences - Future release

While the prebuilt view is powerful, RoomKit recognizes that one size doesn't fit all. For developers seeking a more tailored experience, the **SDK offers granular components, allowing you to craft custom views** based on the state of the room.

This example showcases the ability to switch between different screens based on the room's state so that you can have your own leave screen, offering a dynamic user experience where you can mix and match prebuilt components with your own views.

```swift
import HMSRoomKit
import HMSRoomModels

struct ContentView: View {
    @ObservedObject var conferenceRoom = HMSRoomModel(roomCode: "qfw-mit-seb")
    
    var body: some View {
        Group {
            switch conferenceRoom.roomState {
            case .none:
                HMSPreviewScreen()
            case .meeting:
                HMSConferenceScreen()
            case .leave:
                // code for your own leave screen
            }
        }
        .environmentObject(conferenceRoom)
        .environmentObject(HMSUITheme())
    }
}
```

### Ultimate Customizability - Future release

Taking customization a step further, RoomKit allows developers to not only switch views based on room states but also to intricately design the UI components within those views.

In this example, we see a custom layout for the **meeting** state, where the developer has designed a control bar for **mute, video toggle, and leave actions** while still using the **peer layout grid** from RoomKit. This level of customizability ensures that developers have **complete control over the user experience**.

```swift
import HMSRoomKit
import HMSRoomModels

struct ContentView: View {
    @ObservedObject var conferenceRoom = HMSRoomModel(roomCode: "qfw-mit-seb")
    
    var body: some View {
        Group {
            switch conferenceRoom.roomState {
            case .none:
                HMSPreviewScreen()
            case .meeting:
                VStack {
                    HMSPeerLayout()
                    HStack(spacing: 30) {
                        Spacer()
                        Image(systemName: conferenceRoom.isMicMute ? "mic.slash" : "mic")
                            .onTapGesture { conferenceRoom.toggleMic() }
                        Image(systemName: conferenceRoom.isCameraMute ? "video.slash" : "video")
                            .onTapGesture { conferenceRoom.toggleCamera() }
                        Image(systemName: "phone.down.fill")
                            .onTapGesture {
                                Task { try await conferenceRoom.leave() }
                            }
                        Spacer()
                    }
                    .padding()
                }
            case .leave:
                HMSEndCallScreen()
            }
        }
        .environmentObject(conferenceRoom)
        .environmentObject(HMSUITheme())
    }
}
```

The following video shows the UI of the app using the code above:


<video loop="true" controls="controls" id="vid">
    <source src="https://storage.googleapis.com/100ms-cms-prod/cms/HMS_Prebuilt_Mobile_Code_3_635cd71fd2/HMS_Prebuilt_Mobile_Code_3_635cd71fd2.mp4?updated_at=2023-09-29T07:33:17.554Z" type="video/mp4" />
</video>

## Conclusion
`HMSRoomKit`'s blend of simplicity and customizability ensures that developers, regardless of their requirements, can craft video-conferencing experiences with ease and precision. Whether it's a quick setup or a tailored design, RoomKit delivers, leveraging the power of SwiftUI's declarative paradigm. 

Our Prebuilt roadmap is brimming with exciting plans ahead. If you would like to be part of this journey and contribute to it, do consider joining our [#Prebuilt discord channel](https://discord.gg/jD94Fp74Ea). 
Until next time 👋🏻


thumbnail_Prebuilt_iOS_coverImage.png

Prebuilt_iOS_coverImage.png

Meet all new 100ms RoomKit SDK on iOS

One of the most requested features from our customers was the ability to create polls and quizzes. We are excited to announce that this feature is now available! 

Polls are a great way to get feedback from your audience, learn more about their interests, and engage them in a two-way conversation. Want to get feedback on the session? Create a poll. Let your audience rate the session as good or bad.

The questions can be made single-choice or multiple-choice. The feedback example discussed above would be a single-choice question. Whereas if you want to conduct market research on say what social media platform everyone in the room uses. That would require a multiple-choice question.

Needless to say, polls can enhance audience engagement and add interactivity to virtual meetings, conferences, or live streams. They also enable you to collect well-organized data by posing direct questions.

Create polls in 100ms rooms in the following way:

<video width="1280" height="720" controls> <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Polls_demo_creating_a_poll_using_web_app_0ab6eea062/Polls_demo_creating_a_poll_using_web_app_0ab6eea062.mp4?updated_at=2023-09-05T09:04:36.400Z" type="video/mp4"> </video>

Once a poll is created and launched by a peer, other peers with permission to view the polls can interact with it. 

<video width="1280" height="720" controls> <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Casting_vote_demo_6e3e936feb/Casting_vote_demo_6e3e936feb.mp4?updated_at=2023-09-05T09:04:31.131Z" type="video/mp4"> </video>

Moreover, the Polls can be extended to create quizzes as well. 

To delve a bit deeper, quizzes operate on a similar principle to polls, albeit with a twist. 

In the world of quizzes, questions come with designated correct and incorrect answers, adding an element of assessment and knowledge evaluation to the interactive experience. It can be used to craft educational assessments, test comprehension, and promote active learning through engagement.

Let’s get into the mix of building things using 100ms Polls and Quizzes.

## How to enable polls and quizzes for your rooms on the 100ms dashboard?

1. **Access the 100ms Dashboard**: Begin by visiting the [100ms dashboard](https://dashboard.100ms.live/dashboard).
2. **Choose or Create a Template**: Select the template where you want to enable polls and quizzes. Alternatively, create a new template by clicking the 'Create Template' button.
3. **Access Template Configuration**: Click on the 'Configure' button within your chosen template.
4. **Assign Poll Creation Privileges**: Select which role(s) should have the ability to create polls.
5. **Set Permissions**: In the 'Permissions' section, activate the following toggles:
    ◦ 'Create polls and quizzes'
    ◦ 'Read polls and quizzes'
6. **Exclusive Interaction**: If certain role(s) should only participate in polls without creating them, enable only the 'Read polls and quizzes' toggle for those roles.

<video width="1280" height="720" controls> <source src="https://storage.googleapis.com/100ms-cms-qa/cms/enable_polls_and_quizes_b6b1901149/enable_polls_and_quizes_b6b1901149.mp4?updated_at=2023-09-13T06:55:02.593Z" type="video/mp4"> </video>

## How to add Polls to React sample app?

Let’s add polls to a React sample app to see how easy it is.

Start by creating a new React project (I’ll use Vite to create one) by running the following command:

```jsx
npm create vite@latest my-polls-app -- --template react
```

This would create a new project folder `my-polls-app` in your desired directory with React configured in it.

Now, to quickly add the 100ms React SDK and enable live audio-video in our app, we will follow the React Quickstart Guide [here](https://www.100ms.live/docs/javascript/v2/quickstart/react-quickstart).

Once done, we should have a React project with audio-video calling using 100ms set up.

> The complete React Quickstart Guide code can be found [here](https://codesandbox.io/s/github/100mslive/100ms-examples/tree/main/web/react-quickstart). 

Create a new folder in the `src` directory called `Components` and add two subfolders `Poll` and `UI` to it. These folders would have the code for the necessary UI and Poll components.

We start by creating a `Modal.jsx` file inside the `UI` folder. This file would have the following code: 

```jsx
import { Fragment } from "react";
import ReactDOM from "react-dom";
import classes from "./Modal.module.css";

const Backdrop = (props) => {
  return <div className={classes.backdrop} onClick={props.onClose}></div>;
};

const ModalOverlay = (props) => {
  return (
    <div className={classes.modal}>
      <div className={classes.content}>{props.children}</div>
    </div>
  );
};

const portalElement = document.getElementById("overlays");

const Modal = (props) => {
  return (
    <Fragment>
      {ReactDOM.createPortal(
        <Backdrop onClose={props.onClose} />,
        portalElement
      )}
      {ReactDOM.createPortal(
        <ModalOverlay>{props.children}</ModalOverlay>,
        portalElement
      )}
    </Fragment>
  );
};
export default Modal;
```

 The Modal component is made up of the `Backdrop` and `ModalOverlay` . The `ModalOverlay` would provide a white background to lay the children on passed to it using props.

You’d notice we have imported a css file but haven’t defined it yet. So, inside the same `UI` folder create a new file called `Modal.module.css` and add the following to it: 

```css
.backdrop {
  position: fixed;
  top: 0;
  left: 0;
  width: 100%;
  height: 100vh;
  z-index: 20;
  background-color: rgba(0, 0, 0, 0.75);
}

.modal {
  position: fixed;
  top: 20vh;
  left: 5%;
  width: 90%;
  background-color: #546e7a;
  padding: 1rem;
  border-radius: 14px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.25);
  z-index: 30;
  animation: slide-down 300ms ease-out forwards;
}

@media (min-width: 768px) {
  .modal {
    width: 40rem;
    left: calc(50% - 20rem);
  }
}

@keyframes slide-down {
  from {
    opacity: 0;
    transform: translateY(-3rem);
  }
  to {
    opacity: 1;
    transform: translateY(0);
  }
}
```

Lastly, to complete our Modal, we also need to modify the `index.html` in the root of the project directory to add another `div` with `id="overlays"` before the root `div` as below:

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>100ms Polls Demo</title>
  </head>
  <body>
    <div id="overlays"></div>
    <div id="root"></div>
    <script type="module" src="/src/main.jsx"></script>
  </body>
</html>
```

Now, we will work on a form to create a poll. Start by creating a form as follows:

```jsx
import Modal from ".././UI/Modal";
import { Button } from "@100mslive/roomkit-react";
import { useState } from "react";
import { useHMSActions } from "@100mslive/react-sdk";
import "../../styles.css";

const PollForm = (props) => {
	const hmsActions = useHMSActions();
  const [inputs, setInputs] = useState({});

  return (
    <Modal onClose={props.onClose}>
      <form onSubmit={handleSubmit}>
        <label>
          Enter a Name for the Poll:
          <div className="input-container">
            <input
              type="text"
              name="name"
              value={inputs.name || ""}
              onChange={handleChange}
            />
          </div>
        </label>
        <label>
          Enter a question for the Poll:
          <div className="input-container">
            <input
              type="text"
              name="text"
              value={inputs.text || ""}
              onChange={handleChange}
            />
          </div>
        </label>
        <label>
          Enter the First Value:
          <div className="input-container">
            <input
              type="text"
              name="first"
              value={inputs.first || ""}
              onChange={handleChange}
            />
          </div>
        </label>
        <label>
          Enter the Second Value:
          <div className="input-container">
            <input
              type="text"
              name="second"
              value={inputs.second || ""}
              onChange={handleChange}
            />
          </div>
        </label>
        <input type="submit" />
      </form>
    </Modal>
  );
};
export default PollForm;
```

You’ll notice we are having the user input a name for the poll, a question and two options to choose from. To manage changes in values and successfully submit the form, we add the following functions.

```jsx
const handleChange = (event) => {
    const name = event.target.name;
    const value = event.target.value;
    setInputs((values) => ({ ...values, [name]: value }));
  };

  const handleSubmit = async (event) => {
    event.preventDefault();
    const id = Date.now().toString();
    await hmsActions.interactivityCenter
      .createPoll({
        id,
        title: inputs.name,
        type: "poll",
        rolesThatCanViewResponses: ["host"],
      })
      .then(() => handleCreate(id))
      .catch((err) => console.log(err.message));
  };

const handleCreate = async (id) => {
    console.log("POLL CREATED with ${id}");
    await hmsActions.interactivityCenter.addQuestionsToPoll(id, [
      {
        text: inputs.text,
        type: "single-choice",
        options: [
          {
            text: inputs.first,
            isCorrectAnswer: false,
          },
          {
            text: inputs.second,
            isCorrectAnswer: false,
          },
        ],
        skippable: true,
      },
    ]);
    await hmsActions.interactivityCenter.startPoll(id);
  };
```

The `handleChange` takes any changes in the text input fields and sets the `inputs` variable accordingly. On form submission, the `handleSubmit` is called which uses the `useHMSActions` hook to create a new poll.

Once a poll is created, `handleCreate` is run to `addQuestionsToPoll` and also `startPoll` using the same `hmsActions`.

The complete `PollForm.jsx` file would then look as [this](https://github.com/adityathakurxd/live-audio-video/blob/main/src/Components/Poll/PollForm.jsx).

Next, we return to our `App.jsx` file to display the `PollForm` when the user clicks on a `Create Poll` button.

Start by importing the following:

```jsx
import { useState } from "react";
import { Button } from "@100mslive/roomkit-react";
```

We will use `useState` to manage the state of the `PollForm` if we need to show or hide it.

```jsx
const [pollFormIsShown, setPollFormIsShownn] = useState(false);

const showPollFormHandler = () => {
    setPollFormIsShownn(true);
  };

const hidePollFormHandler = () => {
    setPollFormIsShownn(false);
	};
```

Add the `PollForm` component and a button with `showPollFormHandler` being passed to it as below:

```jsx
{pollFormIsShown && <PollForm onClose={hidePollFormHandler} />}
<Button onClick={showPollFormHandler}>Create Poll</Button>
```

Once this is done, we should be able to toggle and view the `PollForm` and dismiss it by clicking the backdrop area. We should also be able to create and start a poll with the name, question and choices entered by the user.

Let us now work to show the Poll to any other users using our app.

We can use the `useHMSNotifications` hook to be notified when a new poll is started. We will use this to show a toast using `react-toastify` when a new poll has begun.

Add the package by running: `npm install --save react-toastify`

Add the following imports to the `App.jsx` file.

```jsx
import {
  selectIsConnectedToRoom,
  selectLocalPeerID,
  useHMSActions,
  HMSNotificationTypes,
  useHMSStore,
  useHMSNotifications,
} from "@100mslive/react-sdk";
import { ToastContainer, toast } from "react-toastify";
import "react-toastify/dist/ReactToastify.css";
import { useEffect, useState } from "react";
```

 Initialise notifications and localPeerID to use them as follows:

```jsx
const notification = useHMSNotifications();
const localPeerID = useHMSStore(selectLocalPeerID);
```

Using `useEffect` we check for any new notifications. Add the following code to `App.jsx` 

```jsx
const [pollNotificationData, setPollNotificationData] = useState();

useEffect(() => {
    if (!notification) {
      return;
    }
    switch (notification.type) {
      case HMSNotificationTypes.POLL_STARTED:
        if (notification.data.startedBy !== localPeerID) {
          console.log("NOTIFICATION RECEIVED");
          console.log(notification.data);
          setPollNotificationData(notification.data);
          toast(`A new Poll is available: ${notification.data.title}!`);
        }
        break;
      default:
        break;
    }
  }, [notification]);
```

Lastly, add `<ToastContainer />` to the return of `App.jsx` to view a toast when some other remote user creates a new poll. Notice how we have added a check to not show toast to the local peer itself.

Now, we want to be able to cast our vote on the created poll. To do that we again start by showing a modal with the poll data for the users to vote. To the `App.jsx` add the following:

```jsx
const [pollModalIsShown, setPollModalIsShown] = useState(false);

const showPollModalHandler = () => {
    setPollModalIsShown(true);
  };
```

In the return, add a `ViewPoll` component as follows:

```jsx
{pollModalIsShown && (
            <ViewPoll pollNotificationData={pollNotificationData} />
          )}
```

Let us now work on this component. Create a new file called `ViewPoll.jsx` inside the `Poll` folder in the `Components` directory. First, create a basic form layout with radio buttons using the same `Modal` component we had created earlier.

```jsx
import Modal from "../UI/Modal";
import { useState } from "react";
import { Button } from "@100mslive/roomkit-react";
import { useHMSActions } from "@100mslive/react-sdk";
import { toast } from "react-toastify";
import "react-toastify/dist/ReactToastify.css";

const ViewPoll = (props) => {

  return (
    <Modal onClose={props.onClose}>
      <h1>Poll: {props.pollNotificationData.title}</h1>

      <h3>{props.pollNotificationData.questions[0].text}</h3>

      <form onSubmit={handleSubmit}>
        <div className="radio">
          <label>
            <input
              type="radio"
              value={props.pollNotificationData.questions[0].options[0].index}
              checked={Number(selectedOptionIndex) === 1}
              onChange={handleChange}
            />
            {props.pollNotificationData.questions[0].options[0].text}
          </label>
        </div>

        <div className="radio">
          <label>
            <input
              type="radio"
              value={props.pollNotificationData.questions[0].options[1].index}
              checked={Number(selectedOptionIndex) === 2}
              onChange={handleChange}
            />
            {props.pollNotificationData.questions[0].options[1].text}
          </label>
        </div>

        <br />
        <Button type="submit">Submit</Button>
      </form>
    </Modal>
  );
};
export default ViewPoll;
```

And, lastly, add functions to handle changes made in the form and submission as follows:

```jsx
  const actions = useHMSActions();
  const [selectedOptionIndex, setSelectedOptionIndex] = useState();

  function handleChange(event) {
    setSelectedOptionIndex(event.target.value);
  }

  const handleSubmit = async (event) => {
    event.preventDefault();

    await actions.interactivityCenter.addResponsesToPoll(
      props.pollNotificationData.id,
      [
        {
          questionIndex: props.pollNotificationData.questions[0].index,
          option: Number(selectedOptionIndex),
        },
      ]
    );

    toast(`Vote done!`);
  };
```

Notice how we are using the `useHMSActions` hook to `addResponsesToPoll` by passing the poll id obtained from the data passed in as props. 

With this done, our application should now be ready to test!

<video width="1280" height="720" controls> <source src="https://storage.googleapis.com/100ms-cms-qa/cms/Polls_in_action_on_React_sample_124f7fca5f/Polls_in_action_on_React_sample_124f7fca5f.mp4?updated_at=2023-09-05T09:51:50.410Z" type="video/mp4"> </video>

> The code for this project is available on GitHub [here](https://github.com/adityathakurxd/live-audio-video).

The React app that we’ve built here allows you to create the poll, add single-choice questions as well, and start it for remote users. Using the 100ms SDK, we can also create multiple-choice polls and quizzes as mentioned in the article before.

With the ability to curate customized polls, incorporate thought-provoking questions, and initiate real-time participation from remote users through the React app, a world of engaging possibilities opens up. The dynamic fusion of technology and user interaction empowers developers to not only enhance the virtual experience but also extract valuable data and insights.

Ready to dive in? To grasp the full extent of what's achievable and delve into practical steps, look at our comprehensive documentation.

- Get started on the web [here](https://www.100ms.live/docs/javascript/v2/how-to-guides/build-interactive-features/polls).
- Add Polls to your [Android](https://www.100ms.live/docs/android/v2/how-to-guides/interact-with-room/room/polls) or [iOS](https://www.100ms.live/docs/ios/v2/how-to-guides/interact-with-room/room/polls) app.
- Join our [discord server](https://discord.com/invite/kGdmszyzq2) in case of any doubts and to help us build it further.

thumbnail_Quiz.png

Quiz.png

Engage Your Audience with Interactive Polls and Quizzes - A Step-by-Step Guide

Building fast versus the perfect custom setup for your users is a debate that has plagued engineers and PMs alike. And then 100ms Prebuilt was born
🌅.

## 💨 Shipping faster with Prebuilt

Building audio and video experiences should not be difficult, and it should not take more than 30 minutes to set up a video call within your application. Unfortunately, developers often spend days building even a basic video call setup on their platforms. To solve for this, we launched 100ms Prebuilt which abstracts the complexities of audio/video integration, providing your end-users with a reliable live video experience, while managing audio/video and room controls with 100ms dashboard templates.

### 🔗 Prebuilt Links as a way to get started

To make it easier to get started, we have partly solved this problem with [**Prebuilt links**](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/overview#prebuilt-links). These links empower developers to effortlessly generate and share role-specific links tailored for end-users, enabling them to seamlessly join a room. These links can be embedded using a simple iFrame or utilized as a web view in mobile applications.

However, if your use case or, more importantly, your users demand a more seamless video calling experience, it becomes crucial to natively integrate that into your app. Recognizing this need, we've expanded Prebuilt support across various platforms in the form of a native, embeddable component known as `HMSPrebuilt`.

> 💡 **Prebuilt is now available cross-platform**
>
> - Embed Prebuilt as a native component on React, Web, Flutter, iOS, and Android.
> - This component is getting an upgrade with improved UI and customization. [Read more→](#%F0%9F%9A%80-prebuilt-upgrade)

</aside>

### 🌟 HMSPrebuilt component for a more immersive experience

An embeddable, highest-level UI component of a 100ms room, covering all screens and audio-video functionalities, chat, participant list, video layouts, roles, and more.

- This component is shipped through `RoomKit` packages. In the future, we plan to extend more components to build custom experiences with Prebuilt components like Chat, Participant List, Video Layouts, etc. so you can build custom experiences using our Prebuilt components
- Like Prebuilt links, the HMSPrebuilt Component requires room codes to get initialized.

**Step-by-Step Guides -** Follow these guides for [Web](https://www.100ms.live/docs/javascript/v2/quickstart/prebuilt-quickstart), [React](https://www.100ms.live/docs/react-native/v2/quickstart/prebuilt), [Android](https://www.100ms.live/docs/android/v2/quickstart/quickstart), [Flutter](https://www.100ms.live/docs/flutter/v2/quickstart/quickstart)

Let us now see how easy it is to use Prebuilt as a native embed in your apps.

For this, we’ll use the example of a teleconsultation app built using Next.js (a React framework) that lets you connect with medical practitioners. The final application, with HMSPrebuilt component as embed, can be tried out [here](https://consult-doc.vercel.app/) and it would look this: 
 ![](https://storage.googleapis.com/100ms-cms-prod/cms/health_82abbaaeba/health_82abbaaeba.png) 

> The code sandbox for the project is available [here](https://codesandbox.io/p/github/adityathakurxd/100ms-prebuilt-embed-demo/start-with-prebuilt?file=/app/page.js:55) for you to try along.
> 

The steps are as follows:

- Start by adding the `roomkit-react` dependency to your project.
    
    `npm install @100mslive/roomkit-react --save`
    
- Import the dependency in the `app/page.js` file.
    
    `import { HMSPrebuilt } from "@100mslive/roomkit-react";`
    
- Find the comment with the text ‘TODO: Embed HMSPrebuilt component here’ and paste the following with your room code as below:
    
    `<HMSPrebuilt roomCode="<room code>" />` 
    

> Room code can be found on the 100ms dashboard. We have written about **[Room codes for Prebuilt links](https://www.100ms.live/blog/join-links-with-room-codes#room-codes-for-prebuilt-links)** in an earlier piece.
> 

That is it! We should now be able to see an audio-video calling option on our landing page.

The final project code is available on [GitHub](https://github.com/adityathakurxd/100ms-prebuilt-embed-demo) and [CodeSandbox](https://codesandbox.io/p/github/adityathakurxd/100ms-prebuilt-embed-demo/main?layout=%257B%2522sidebarPanel%2522%253A%2522SEARCH%2522%252C%2522rootPanelGroup%2522%253A%257B%2522direction%2522%253A%2522horizontal%2522%252C%2522contentType%2522%253A%2522UNKNOWN%2522%252C%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522id%2522%253A%2522ROOT_LAYOUT%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522UNKNOWN%2522%252C%2522direction%2522%253A%2522vertical%2522%252C%2522id%2522%253A%2522clm60u4m7000j3b6n3dwfza3x%2522%252C%2522sizes%2522%253A%255B70%252C30%255D%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522EDITOR%2522%252C%2522direction%2522%253A%2522horizontal%2522%252C%2522id%2522%253A%2522EDITOR%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522EDITOR%2522%252C%2522id%2522%253A%2522clm60u4m7000e3b6n0ww6ojmk%2522%257D%255D%252C%2522sizes%2522%253A%255B100%255D%257D%252C%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522SHELLS%2522%252C%2522direction%2522%253A%2522horizontal%2522%252C%2522id%2522%253A%2522SHELLS%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522SHELLS%2522%252C%2522id%2522%253A%2522clm60u4m7000h3b6nd5tddwr5%2522%257D%255D%252C%2522sizes%2522%253A%255B100%255D%257D%255D%257D%252C%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522DEVTOOLS%2522%252C%2522direction%2522%253A%2522vertical%2522%252C%2522id%2522%253A%2522DEVTOOLS%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522DEVTOOLS%2522%252C%2522id%2522%253A%2522clm60u4m7000i3b6nw4ts5blj%2522%257D%255D%252C%2522sizes%2522%253A%255B100%255D%257D%255D%252C%2522sizes%2522%253A%255B50%252C50%255D%257D%252C%2522tabbedPanels%2522%253A%257B%2522clm60u4m7000e3b6n0ww6ojmk%2522%253A%257B%2522id%2522%253A%2522clm60u4m7000e3b6n0ww6ojmk%2522%252C%2522activeTabId%2522%253A%2522clm60xu4200bh3b6netly6ygz%2522%252C%2522tabs%2522%253A%255B%257B%2522id%2522%253A%2522clm60u4m7000d3b6nh53rsfjf%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522FILE%2522%252C%2522filepath%2522%253A%2522%252FREADME.md%2522%252C%2522state%2522%253A%2522IDLE%2522%257D%252C%257B%2522type%2522%253A%2522FILE%2522%252C%2522filepath%2522%253A%2522%252Fapp%252Fpage.js%2522%252C%2522id%2522%253A%2522clm60xu4200bh3b6netly6ygz%2522%252C%2522mode%2522%253A%2522temporary%2522%252C%2522state%2522%253A%2522IDLE%2522%257D%255D%257D%252C%2522clm60u4m7000i3b6nw4ts5blj%2522%253A%257B%2522id%2522%253A%2522clm60u4m7000i3b6nw4ts5blj%2522%252C%2522tabs%2522%253A%255B%257B%2522type%2522%253A%2522TASK_PORT%2522%252C%2522taskId%2522%253A%2522dev%2522%252C%2522port%2522%253A3000%252C%2522id%2522%253A%2522clm625ras00gz3b6ot9ofgkft%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522path%2522%253A%2522%252F%2522%257D%255D%252C%2522activeTabId%2522%253A%2522clm625ras00gz3b6ot9ofgkft%2522%257D%252C%2522clm60u4m7000h3b6nd5tddwr5%2522%253A%257B%2522tabs%2522%253A%255B%257B%2522id%2522%253A%2522clm60u4m7000f3b6n6i0pme7l%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522TASK_LOG%2522%252C%2522taskId%2522%253A%2522dev%2522%257D%252C%257B%2522id%2522%253A%2522clm60u4m7000g3b6no98y2nsr%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522TERMINAL%2522%252C%2522shellId%2522%253A%2522clm4skzmm000we3fq3sheczhu%2522%257D%255D%252C%2522id%2522%253A%2522clm60u4m7000h3b6nd5tddwr5%2522%252C%2522activeTabId%2522%253A%2522clm60u4m7000f3b6n6i0pme7l%2522%257D%257D%252C%2522showDevtools%2522%253Atrue%252C%2522showShells%2522%253Atrue%252C%2522showSidebar%2522%253Atrue%252C%2522sidebarPanelSize%2522%253A15%257D).


### 🎨 Custom UI versus HMSPrebuilt component

Building customized video experiences is undeniably appealing, but it can be quite demanding on your development resources. 100ms SDKs are powerful and highly extensible to build and support all custom experiences and UI.

However, if you find yourself unsure about committing to an extensively customized video experience, and what you truly need is a speedy, low-code, and customizable (hint: Big Update coming soon) solution that doesn't demand a substantial effort, then HMSPrebuilt might be your solution of choice.

## 🚀 Prebuilt Upgrade

In our upcoming September'23 release, 100ms Prebuilt is getting some major upgrades. The idea is to reduce the development time and effort and provide a customizable solution that helps you go live faster.

![prebuilt ui](https://storage.googleapis.com/100ms-cms-qa/cms/Prebuilt_87264fddcd/Prebuilt_87264fddcd.png?updated_at=2023-09-12T12:20:06.795Z)

You'll be able to do more with Prebuilt. We are coming out with the following: 

- **Ready-to-ship UI** built for use cases: We're upgrading the calling experience with improved UI/UX, including room controls and beautiful video layouts for 1:1, group calls, and live streaming, all with good defaults built-in.
- **Prebuilt Customiser**: Customise everything from customizing Prebuilt to match your themes, setting up a custom welcome text to configuring chat and participant list controls, and much more.
- You can get the upgraded experience on the Prebuilt Links and `HMSPrebuilt` Component after this release.

This is just the first of many releases to come. Watch out for more of our upcoming blogs which will provide more detail on this upgrade. We have a lot more planned in our Prebuilt roadmap and if you'd like to be a part of this journey and contribute to it, please tune in to our [#Prebuilt discord channel](https://discord.gg/jD94Fp74Ea).

thumbnail_Embed Prebuilt - 1.png

Embed Prebuilt .png

Embedding Prebuilt as a native component 

Live classrooms have become the norm for learning ever since the pandemic. A teacher presents the class while students actively listen and ask questions. That is the usual story. However, it is not the case in countries like India where students from tier 2 and tier 3 cities face network issues. In this blog, we will talk about how 100ms solves this problem for EdTech large classrooms by using HLS and WebRTC technologies simultaneously.

## Problem with the current stack

When it comes to live classrooms, WebRTC is commonly considered the optimal solution. WebRTC is a real-time communication protocol that enables users to interact with audio-video capabilities while maintaining low latency. This is the same technology that powers one-on-one video calls and video conferences in apps like Google Meet, Skype,…

While WebRTC might seem like a no-brainer answer to EdTech live classrooms, it is not without its own flaws. There are mainly two cases where audio-video issues start to become visible:

1. **Bad network**: When the users on the call are experiencing a poor network connection, which is super common in countries like India.

2. **High Scale**: When there are too many users on the call.

In the context of live classrooms, scalability aside, the users on bad network recurrently face issues like blurred content on video, skipped or black frames, and choppy audio:

1. **Blurred content on video** - The text content and figures being shown during classes look blurred and the overall video quality is subpar.

2. **Skipped or Black frames** - Video looks stuck, fast forwards or goes black during the class suddenly.

3. **Choppy Audio** - Unclear or breaking audio.

> There are apps like Zoom, that use a modified version of WebRTC. However, they still suffer from the same problems that come with WebRTC and, in some cases, a few extra ones.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/live_classroom_webrtc_hls_usual_stack_demo_913b79d574/live_classroom_webrtc_hls_usual_stack_demo_913b79d574.gif) 

## A tale of latency and quality

Before we move on further, it’s essential to understand the two key factors to consider when selecting a technology for live video use cases: latency and audio/video quality.

1. **Latency**: Latency refers to the time it takes for media to travel from the source to the receiver. Low latency is essential for real-time communication, enabling seamless interaction between participants.

2. **Audio/video quality**: Video quality is determined by factors such as resolution, frame rate, and bitrate. Higher quality visuals contribute to a more engaging and immersive learning experience.

An ideal solution would offer both low latency and high video quality. However, in practice, it often involves a trade-off. Either prioritize low latency, sacrificing video quality to ensure real-time communication or focus on delivering high video quality but with increased latency.

### WebRTC - low latency, subpar quality

WebRTC, designed for two-way communication, aims to minimize latency. However, this focus on low latency can result in compromises in video quality. Some examples of how WebRTC compromises on video quality include:

1. **Packet loss and frame drops**: Network limitations, such as limited bandwidth or high latency, can lead to packet loss and dropped frames. These issues can cause a degraded video experience, impacting the clarity and smoothness of the visuals.

2. **Adaptive bitrate streaming challenges**: WebRTC employs adaptive bitrate streaming to adjust video quality dynamically based on network conditions. However, the adaptive streaming algorithm may not respond quickly enough to network changes, leading to temporary compromises in video quality.

3. **Reduced resolution and compression artifacts**: In low-bandwidth situations, WebRTC may dynamically adjust the bitrate and compression settings of the video stream. This can lead to a reduction in video resolution and the introduction of compression artifacts which can lead to a compromise in the level of detail, affecting the overall quality of the video.

WebRTC as a protocol was not designed for scalability, but rather for peer-to-peer communication. It was later adopted to support multiple peers in a call. Therefore, scaling a WebRTC call is not an easy task and, after a certain point, it’s just a question of making further compromises.

### HLS - high latency, high quality

In live streaming use cases where the users are only on the receiving end, the most popular technology is HLS. It is the same technology that powers Super Bowl and IPL match live streams at scale. Unlike WebRTC, this comes with a latency of ~8 seconds but offers the best video quality possible. The latency, however, can be further reduced through optimizations or by moving to LL-HLS at a later stage. HLS offers certain advantages over WebRTC in terms of video quality:

1. **No Frame or Packet Loss**: HLS is designed for reliable delivery and ensures minimal frame or packet loss during transmission. It divides the content into small, individual files, allowing viewers to download segments independently. This approach mitigates the risk of losing an entire video frame or packet during the streaming process.
2. **Content caching and CDN support**: HLS leverages standard HTTP infrastructure and is compatible with Content Delivery Networks (CDNs). CDNs can cache HLS segments, reducing the load on the origin server and improving video delivery performance. This optimized content delivery contributes to better video quality and reduces the potential impact of network congestion.
3. **Scalability**: HLS through use of CDNs, can efficiently deliver content to a large audience (on the scale of millions) without straining server resources. 

In the next section, we will explore how 100ms utilizes HLS and WebRTC technologies simultaneously to address the network issues students face in large EdTech classrooms, bringing the best of both worlds.

## Enhancing live classroom experiences

Let’s consider an elaborate example of a live classroom scenario with a tutor and a group of students. By default, the students are muted and can only interact through chat messages. If necessary, they can raise their hand, and once accepted by the tutor, they can come “on stage” and engage with audio and video enabled. When they have finished interacting, the tutor can make them go “off stage” like the rest of the students. There are three types of users in the call:

1. **Tutor** - The tutor who has admin access over the entire call, with audio and video capabilities.

2. **Students** - Students who can listen to the tutor and interact through chat messages, raise their hand, and so on.

3. **On-stage participants** - Students who have audio and video capabilities to interact with the tutor in near real-time.

### Live classroom with WebRTC and HLS

The tutor joins in a WebRTC connection which is then used to live stream on HLS. By default, the students join as HLS viewers, resembling the experience of watching a class via YouTube Live stream in [Low latency mode](https://support.google.com/youtube/answer/7444635). The only difference in this setup is that students watching the live stream can become part of the stream and interact with the tutor anytime they want.

Any number of students can raise their hand and transition to become on-stage participants, moving from HLS to the WebRTC connection for direct interaction with the tutor. During this transition, the on-stage participant loses 8 seconds of the live stream. These on-stage participants become part of the HLS stream alongside the tutor, allowing their visibility to the remaining students. Once their interaction concludes, they can go back to being HLS viewers. To prevent them from viewing the 8-second time period of live stream during their presence, a timer can be displayed instead.

<video width="1280" height="720" controls>
  <source src="https://storage.googleapis.com/100ms-cms-qa/cms/live_classroom_webrtc_hls_flow_f7f4c7414f/live_classroom_webrtc_hls_flow_f7f4c7414f.mp4?updated_at=2023-07-31T08:09:40.199Z" type="video/mp4">
</video>

Here’s the on-stage participant flow:

1. At time T, the student is watching the HLS stream, while the tutor is at time T+8 seconds in the WebRTC connection.
2. At time T+8 seconds, the tutor asks students to raise their hands if they have any doubts. Students have an 8-second window to raise their hand.
3. At time T, a student raises their hand, and the tutor is notified.
4. Upon the tutor's approval, the student becomes an on-stage participant, transitioning from HLS to the WebRTC connection.
5. The student, who was at time T (in the HLS stream), moves to the WebRTC connection at T+8 seconds. This implies that the student misses 8 seconds of the tutor's time and jumps to T+8 seconds.
6. The student who became the on-stage participant engages in interaction with the tutor, and once finished, returns to being an HLS viewer.
7. When the on-stage participant reverts to being an HLS viewer, they jump to T seconds, aligned with other students. Now, the student has to wait 8 seconds continuing the live stream to avoid watching their own presence.

## Next Steps

Are you interested in building a live classroom using the WebRTC+HLS setup, but don't want to worry about protocols and optimizations? Check out [100ms SDKs](https://www.100ms.live/docs), with built-in support for both WebRTC and HLS. The best part is, switching between WebRTC and HLS is as simple as making a function call.

Feel free to [book a call with us](https://www.100ms.live/contact) to understand more.

thumbnail_live-classroom-webrtc-hls-cover.png

live-classroom-webrtc-hls-cover.png

Enhanced live classroom experience at scale with the WebRTC-HLS stack

thumbnail_WhatsApp (1).png

WhatsApp (1).png

How to build a WhatsApp like audio/video calling in Flutter using CallKit

We're thrilled to announce exciting new features on our platform - **Speaker Labelled Transcription** and **AI-generated Summary**.

Recordings of live video interactions serve as valuable assets enabling on-demand content, quality enhancement, training, feedback, and audit purposes. However, the time-consuming nature of viewing recordings poses a significant challenge.

Transcriptions and summaries help you extract answers and insights faster from recordings. With our new transcription and summary capabilities, we not only convert the human analog form of information into actionable data points, allowing for faster analysis and a comprehensive bird’s eye view, but we also enable *analysis* and *searchability* of the information.

Moreover, we understand the hassle of working with multiple services and integrating various plugins. To streamline this process, we have integrated these capabilities into our recording APIs and developed a user-friendly dashboard UI for a quick implementation. 

If you’ve been a user of 100ms recordings, these new features will fit neatly into your consumption workflow. 

## Transcripts with Speaker Labels

While many services can transcribe videos, understanding who spoke when is crucial for gaining comprehensive context. Speaker labels significantly enhance the quality of information extracted from transcriptions, enabling topical analysis, interests, speaking patterns across all of user’s calls.

Here’s what 100ms’ transcription does well: 

- **Speaker Labelled:** By utilising the underlying audio track of each peer, we accurately identify speakers and map their spoken words. In contrast, most other transcription services use audio characteristics to do speech segmentation and extract embeddings to identify speakers. But this approach is susceptible to issues with factors like number of speakers, audio quality, use of short and filler phrases. Also, even after speaker attribution, it is not possible to identify the exact speaker’s name or user ID in any way.
- Support for **Custom Vocabulary**: This enables you to add non-dictionary words like names, slang, abbreviations, and more that may not be recognised by the AI model.
- Multiple File Formats: You have various file formats to choose from:
    - *Text* file (.txt) for direct consumption.
    - *Subtitle* file (.srt) which allows for synchronised playback.
    - *Granular Structured* (.json) file which associates peer names with peer IDs, provides word and sentence-level granularity with timestamps, making this extremely versatile. It enables mapping all of user's calls, identifying behavioural patterns, conducting sentimental analysis, and gauging live engagement levels for actionable insights.
- Generated **Post Call Completion**: *Room Composite Recordings* currently serve as a prerequisite for accessing transcription and summary features, but we plan to expand this in future releases.

## Call Summaries Generated by AI

Initially skeptical of the value AI summaries could provide, we now heavily rely on them for design discussions and customer interactions. The following summary is generated from the transcript of team's discussion on the initial draft of this blog.

![summary example](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2023_07_13_at_5_23_32_PM_578629b758/Screenshot_2023_07_13_at_5_23_32_PM_578629b758.png?updated_at=2023-07-14T08:19:00.874Z)


We’ve built our first summary feature release to not only summarise the calls in the Minutes of the Meeting format but we’ve also kept it customisable. Here’s what is possible:

- **Customisable Output**: You can,
    - Choose from our pre-defined default **Output Sections;** Agenda, Key Points Covered, Follow-up Action Items, and Short Summary.
    - Add your own questions in the **Output Sections.** This is very helpful in tracking specific inquiries like how many times did John ask “Can we this ship faster?”. 
    - Choose between *bullets* or *paragraph* as **Output Format** for the different sections.
- You can increase the **Model Temperature** to make the AI be more creative or decrease to make it more deterministic and predictable. Higher creativity yields deeper insights, but once in a while, the AI will hallucinate and add imaginary fragments to the summary.
- Provide **Context**: Treat AI as a really smart colleague who has just joined a conversation. Help set context about the conversation and share expectations about the discussion points to output a better summary. 
- Generating the summary requires a transcription as a prerequisite, and the current output format is a JSON file.

## What’s in store for the future?

This is only the beginning for this feature. We have much planned in our future releases.

Here’s a sneak peak:

- Separate transcription and summary APIs which will allow you to transcribe and summarise past recordings.
- Standalone transcription and summary without the need for recording.
- Ability to transcribe multiple roles independently in a single session and have separate summaries for them.

## Enable transcription for your recordings now

We’ve kept this feature completely **free** for now since this is in Beta. We’re looking to gather as much feedback as we can and can’t wait for you to try it out.

We’ve kept it super simple to get started with transcription and summary and written [this neat little guide](https://www.100ms.live/docs/server-side/v2/how-to-guides/enable-transcription-and-summary) to help you with that.

![Enable transcription now](https://storage.googleapis.com/100ms-cms-qa/cms/enable_transcription_c9bb559b68/enable_transcription_c9bb559b68.gif?updated_at=2023-07-14T08:01:29.290Z)

thumbnail_Call AI launch.png

Call AI launch.png

Transcripts and AI-powered summaries, now live in Beta

Karthikeyan S

Following global digitalization trends, KYC (Know Your Customer) has taken the form of a paperless, live video verification process. This vKYC, also known as video KYC, is already live in several countries such as India, Estonia, and Germany, and is already being mandated in some of them.

There are several variations in the process based on the regulatory requirements of the country, but the essence of video KYC is straightforward:

- The user and a KYC agent connect in a live video call. During the call, the agent verifies the user's identity through a live photograph and clear images of their officially valid documents.
- The agent captures these images, and AI-driven Face-Match/Face Liveness checks are performed to further authenticate the user's identity and prevent potential fraud or identity theft.

While there are several video SDKs available in the market to assist with a basic video call setup, this blog explores some use-case-specific features of the 100ms SDK that can facilitate a quick user journey for building vKYC into your app.

## Capture photo/frame at the highest resolution

During verification, the KYC agent occasionally needs to take pictures of the user and their documents. However, attempting to capture a frame from the video stream may not be the best idea. Not only is it difficult to implement, but the quality of the output may not be good enough for verification purposes most of the time.

With 100ms, capturing and saving a high-resolution photo (with a quality similar to the default camera app on the phone) is possible with a simple SDK method call.

![High resolution frame capture](https://storage.googleapis.com/100ms-cms-qa/cms/vkyc_100ms_frame_capture_640cc8aed1/vkyc_100ms_frame_capture_640cc8aed1.gif) 

**Android**

Give a file path to save the image to the method. The image is stored in `JPEG` format by default.

```kotlin
val localVideoTrack : HMSVideotrack = hmsPeer.videoTrack
val cameraControl = (localVideoTrack as? HMSLocalVideoTrack)?.getCameraControl()
...
// capture the image and save it to the specified file path
val imagePath = File(getExternalFilesDir(null), "captured_image.jpeg")
cameraControl.captureImageAtMaxSupportedResolution(imagePath) { isSuccess ->
    if (isSuccess) {
        Log.d(TAG, "Image saved to path: ${imagePath.absolutePath}")
    }
}
```

**iOS**

Get access to `UIImage` instance of the captured image and save it in the format of your wish.

```swift
let localVideoTrack = peer.localVideoTrack()
...
// capture the image
localVideoTrack.captureImageAtMaxSupportedResolution(withFlash: false) { image in
		// create JPEG from `UIImage` and save it the specified file path
		if let imageData = image.jpegData(compressionQuality: 0.8) {
        let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
        let imagePath = documentsPath.appending("/captured_image.jpeg")
				let fileManager = FileManager.default
        fileManager.createFile(atPath: imagePath, contents: imageData, attributes: nil)
				print("Image saved to path: \(imagePath)")
    }
}
```

## Camera controls

Having access to camera controls like tap-to-focus, zoom and flash can help improve the quality of video verification. We will now explore these camera controls in detail and how to enable them with 100ms.

 ![Zoom and Tap-to-focus](https://storage.googleapis.com/100ms-cms-qa/cms/vkyc_100ms_zoom_and_focus_87fca8cc6f/vkyc_100ms_zoom_and_focus_87fca8cc6f.gif) 

### Manual/Tap-to-focus

Sometimes, the subject of a photo might be out of focus, even with the default autofocus of the mobile camera working in the background. This can result in documents appearing blurry during verification. With 100ms, it is possible to capture better quality frames by implementing the tap-to-focus feature.

**Android**

Use a simple `GestureDetectorCompat` instance to map the video view’s coordinates to the SDK method and add support for tap-to-focus.

```kotlin
val localVideoTrack : HMSVideotrack = hmsPeer.videoTrack
val cameraControl = (localVideoTrack as? HMSLocalVideoTrack)?.getCameraControl()
...
val localVideoView : HMSVideoView = findViewById(R.id.local_video)
localVideoView.addTrack(localVideoTrack)
...
// create a gesture detector
val gestureDetector = GestureDetectorCompat(context, object : GestureDetector.SimpleOnGestureListener() {

    override fun onDown(e: MotionEvent?) = true
		// on tap event listener
    override fun onSingleTapUp(event: MotionEvent): Boolean {
				// check if tap-to-focus is supported
        if (cameraControl.isTapToFocusSupported())
						// set focus to the coordinates from tap event
            cameraControl.setTapToFocusAt(
                event.x,
                event.y,
                viewWidth = width,
                viewHeight = height
            )
        return true
    }
})
// attach the gesture detector to the local video view
localVideoView.setOnTouchListener { _, event ->
    var didConsume = gestureDetector.onTouchEvent(event)
    didConsume
}
```

**iOS**

Simply set the `cameraFocusMode` in `HMSTrackSettings` configuration, after initializing the SDK. The tap-to-focus will be enabled on local video view.

```swift
...
// configure track settings
sdk.trackSettings = HMSTrackSettings.build { videoSettingsBuilder, audioSettingsBuilder in
    // set focus mode to `TapToAutoFocus`
		videoSettingsBuilder.cameraFocusMode = .focusModeTapToAutoFocus
    ...
}
```

### Zoom

The user might want to zoom in/out to better view the documents or scope in on something important, as part of the verification. With the 100ms SDK, it is possible to easily implement pinch-to-zoom-like features based on gestures.

**Android**

Use a simple `ScaleGestureDetector` instance to add zoom with pinch gesture, using the SDK methods like this:

```kotlin
val localVideoTrack : HMSVideotrack = hmsPeer.videoTrack
val cameraControl = (localVideoTrack as? HMSLocalVideoTrack)?.getCameraControl()
...
val localVideoView : HMSVideoView = findViewById(R.id.local_video)
localVideoView.addTrack(localVideoTrack)
...
// set `lastZoom` to be the minimum zoom value
var lastZoom = cameraControl.getMinZoom()
// create a scale gesture detector
val scaleGestureDetector = ScaleGestureDetector(
    context,
    object : ScaleGestureDetector.SimpleOnScaleGestureListener() {
        override fun onScale(detector: ScaleGestureDetector): Boolean {
						// check if zoom is supported
            if (cameraControl.isZoomSupported()) {
								// update zoom value based on detected change 
								// in scaleFactor for the pinch gesture
                lastZoom *= detector.scaleFactor
                cameraControl.setZoom(lastZoom)
                return true
            }
            return false
        }
    })
// attach the scale gesture detector to the local video view
localVideoView.setOnTouchListener { _, event ->
    var didConsume = scaleGestureDetector.onTouchEvent(event)
    didConsume
}
```

**iOS**

Use a `UIPinchGestureRecognizer` instance with a callback to implement pinch-to-zoom functionality, like this:

```swift
class ViewController: UIViewController {
    
    let localVideoView = HMSVideoView()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        // add a pinch gesture recognizer to the camera view
        let pinchGesture = UIPinchGestureRecognizer(target: self, action: #selector(handlePinchGesture(_:)))
        localVideoView.addGestureRecognizer(pinchGesture)
    }

    @objc func handlePinchGesture(_ gesture: UIPinchGestureRecognizer) {
        // get access to the video track and make sure it's local video tack
        if let localVideoTrack = localVideoView.videoTrack() as? HMSLocalVideoTrack {
            // modify the underlying camera device to change zoom factor
            localVideoTrack.modifyCaptureDevice { device in
                
                guard let device = device else { return }
                
                let newZoomFactor = device.videoZoomFactor * gesture.scale
                // make sure to not exceed camera zoom range
                let clampedZoomFactor = max(device.minAvailableVideoZoomFactor, min(newZoomFactor, device.activeFormat.videoMaxZoomFactor))
                // set the new zoom factor on camera device
                device.videoZoomFactor =  clampedZoomFactor
            }
        }
    }
}
```

### Flash

In low lighting conditions, a flash light can help identify the subject and the documents clearly. Here’s how you can easily enable/disable the flash with 100ms SDK.

**Android**

Toggle flash based on its enabled status.

```kotlin
val localVideoTrack : HMSVideotrack = hmsPeer.videoTrack
val cameraControl = (localVideoTrack as? HMSLocalVideoTrack)?.getCameraControl()
...
// check if flash is supported
if(cameraControl.isFlashSupported()){
		// toggle flash
		cameraControl.setFlash(enable: !cameraControl.isFlashEnabled())
}
```

**iOS**

Toggle flash based on its enabled status.

```swift
let localVideoTrack = peer.localVideoTrack()
...
localVideoTrack.modifyCaptureDevice { device in

    guard let device = device else { return }
		// check if flash is supported
    guard device.isTorchModeSupported(.on) else { return }
    // toggle flash
    device.torchMode = device.torchMode == .off ? .on : .off
}
```

## Picking the right video SDK

Building a reliable and scalable video infrastructure from scratch is super expensive and time-consuming. For vKYC use cases, the right way to go live is by integrating a highly customizable, reliable and easy-to-integrate [live video solution like 100ms](https://www.100ms.live/). 100ms offers live video SDKs that can help you build high-quality live video experiences—from video conferencing to interactive live streaming, at scale. 

### Compliance and Security

100ms is **SOC 2 (both Type 1 and 2), HIPAA and CERT-In VAPT compliant** when it comes to compliance certifications. From a security perspective, this is what it could mean to its customers:

- Third-party vendor assess the 100ms infrastructure and tech stack once every year for security vulnerabilities.
- Calls and recordings are always encrypted during transmission and storage rendering any unauthorized access useless.
- Customer data isn’t stored without permission, you can choose not to share any data/recordings of the video KYC sessions from your app.

### Reliability and other things…

100ms SDK comes with built-in reliability—from optimizing for bad network conditions to noisy environment, the SDK can handle all the edge cases that degrade call quality.  Do check out [this blog on how 100ms tests for network reliability](https://www.100ms.live/blog/network-reliability-test), that was written before [adaptive bitrate](https://www.100ms.live/docs/concepts/v2/concepts/adaptive-bitrate) was added to improve it. As for call recordings, they go to your own AWS S3 or GCP bucket to ensure that you are in control with a 100% success rate.

There are several other things that can’t be covered in this blog, that make 100ms the best choice for video KYC. For example, **network quality reports** feature can help the agents get real-time network speed of the user. This can also be used to check if the user meets the minimum network speed requirement before proceeding to the verification.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/vkyc_100ms_network_quality_report_410bcdeb78/vkyc_100ms_network_quality_report_410bcdeb78.png) 

To know more in detail about 100ms, check out the [basic concepts of 100ms](https://www.100ms.live/docs/concepts/v2/concepts/basics) and then get started on [Android](https://www.100ms.live/docs/android/v2/get-started/quickstart) and [iOS](https://www.100ms.live/docs/ios/v2/get-started/quickstart) SDKs.

thumbnail_Image.png

Image.png

Why vKYC is more than just a video call

100ms offers interactive live streaming to create two-way, near-real-time interactions between streamers and viewers with features like chat and emoji reactions. Live streaming can scale to very large rooms (with ~10k participants) with a latency of 7-10 seconds.

This latency means interactive experiences that are outside the video stream need to be in sync with the video. **Timed metadata makes this possible.**

For example, a streamer is running a countdown and at the end of this countdown, they want to showcase confetti to their viewers. For maximum impact, this confetti needs to be perfectly aligned with the countdown.

<iframe width="100%" height="380" src="https://www.youtube.com/embed/GS4BXz4xqZ4" title="Timed metadata example" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

<br>
In this article, we see how to enable Timed Metadata and use it in a live streaming example to trigger certain elements on the viewer side synchronised with the broadcaster’s action.

## What is Timed Metadata?

Timed Metadata allows attaching metadata to a specific timestamp in the HLS live stream. This metadata is triggered when the live stream playhead is on the specific timestamp. Thus, enabling information to be perfectly synchronised with the audio and video frames, and ensuring that all viewers receive the metadata at the same moment relative to the stream. The time code acts as a reference point, allowing for triggered actions based on the data, such as:

- Presenting questions for an interactive quiz show streamed live.
- Displaying product information for an e-commerce stream.

And, more uses cases that are discussed further in this blog.

## Supported Devices

The Desktop and Mobile browsers (both on Android and iOS) are supported. 

This differentiates Timed Metadata offered by 100ms from the competitors as they do not support iOS mobile browsers.

You can also integrate it into your [Native Android](https://www.100ms.live/docs/android/v2/how--to-guides/record-and-live-stream/hls/hls-timed-metadata) and [Native iOS](https://www.100ms.live/docs/ios/v2/how--to-guides/record-and-live-stream/hls/hls-timed-metadata) applications.

## Getting Started

For this example, we’ll use a live-streaming example built by [Taminoturoko Briggs](https://blog.logrocket.com/author/taminoturokobriggs/) as the starter code. This would help quickly test out the feature.

> The starter code for the project can be found [here](https://github.com/Tammibriggs/LiveStreaming-100ms).

We start by forking the project and cloning it to our system.

If you’d like to setup live streaming in your React app from scratch, we have detailed documentation available [here](https://www.notion.so/Peer-Metadata-d9042f28237145df8bc5985c18a71334).

**Let’s get started by first setting up the project on the 100ms dashboard!**

<iframe width="100%" height="380" src="https://www.youtube.com/embed/2PY95SxnAuU" title="Setup the live streaming project on the 100ms dashboard" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

### Enable timed metadata

Navigate to Destinations under the Template Settings and enable Timed Metadata as follows:

 ![Enable timed metadata](https://storage.googleapis.com/100ms-cms-qa/cms/enable_timed_metadata_ea4cf9d295/enable_timed_metadata_ea4cf9d295.gif) 

## Adding the Control buttons

After cloning the project above we can start by adding the buttons to trigger timed metadata in the live stream. Add the following code under the ‘Go Live’ button in the `Controls.js` file:

```jsx
         <Button
            variant="contained"
            disableElevation
            onClick={sendTimedMetadata}
          >
            Send metadata
          </Button>
```

This would add a ‘Start Celebration’ button to the broadcaster’s screen. We’ll define the function added to the `onClick` in the next step.

## Send timed metadata

To send timed metadata we create a new function inside the same `Controls.js` file as follows:

```jsx
const sendTimedMetadata = async () => {
    const data = { triggerConfetti: true };
    await hmsActions.sendHLSTimedMetadata([
      {
        payload: JSON.stringify(data),
        duration: 2,
      },
    ]);
  };
```

We call `hmsActions.sendHLSTimedMetadata` and pass the `data` as `payload`.

## Trigger Confetti on Viewer’s Screen

Now, when the Timed Metadata is received we want to trigger confetti on the viewer’s end. For this, we follow the steps below.

- Install the `@100mslive/hls-player` dependency using:

```bash
npm i @100mslive/hls-player
```

- Add the following imports to the file:

```jsx
import { HMSHLSPlayer } from "@100mslive/hls-player";
import { HMSHLSPlayerEvents } from "@100mslive/hls-player";
```

- Next, add the confetti package to your project and import it as:

```jsx
import Confetti from "react-confetti";
```

- Update the `HlsView` function with the following code inside `HlsView.js`

```jsx
function HlsView() {
  const videoRef = useRef(null);
  const hlsState = useHMSStore(selectHLSState);
  const hlsUrl = hlsState.variants[0]?.url;
  const [status, setStatus] = useState(false);
  const [hmsPlayer, setHMSPlayerInstance] = useState(null);

  // initialise HMSPLayer
  useEffect(() => {
    if (videoRef && !hmsPlayer && hlsUrl) {
      const player = new HMSHLSPlayer(
        hlsUrl,
        videoRef.current
      );
      setHMSPlayerInstance(player);
      player.play();
    }
  }, [videoRef, setHMSPlayerInstance, hmsPlayer, hlsUrl]);

  // listen to hls player events
  useEffect(() => {
    if (hmsPlayer) {
      console.log("adding event listener");
      hmsPlayer.on(HMSHLSPlayerEvents.TIMED_METADATA_LOADED, (data) => {
        console.log(data);
        const receivedData = JSON.parse(data.payload);
        console.log(receivedData.triggerConfetti);
        setStatus(receivedData.triggerConfetti);
      });
    }
  }, [hmsPlayer, setStatus, status]);

  return (
    <div>
      {status ? <Confetti width={1200} height={800} /> : null}
      <video ref={videoRef} autoPlay control playsinline></video>;
    </div>
  );
}

export default HlsView;
```

When timed metadata is received, it is parsed and logged on the console. We also `setStatus` to the payload value which is `true` to show confetti on the screen.

> Note: The complete code can be found [here](https://github.com/adityathakurxd/timed_metadata).
> 

With that done, we have updated the Live Streaming example to demo Timed Metadata. When the broadcaster clicks on the ‘Send metadata` button, it would show confetti on the viewer’s end.

This can be used to time celebrations with your viewers such as a New Year's Eve Countdown, or a Product launch at a specified time.

## Timed Metadata Use Cases

Ready to build? Here are some ideas to inspire what you can build with Timed Metadata:

**1. Enhanced Live shopping experience**

Timed Metadata can help enhance the live shopping experience. For example, as the host showcases a product, timed metadata can be used to prompt the user to add the product to the cart outside of the video stream and complete the purchase. This would help create an immersive and interactive shopping experience for viewers, and increase the chances of converting them into customers.

**2. Names and profiles of speakers in a live panel discussion:** 

During a live panel discussion or lecture, timed metadata can be used to display the names and titles of the speakers, along with other details, that can be made clickable outside the video stream. This would them to access the speakers' profiles, websites, social media handles, or other relevant links with just a click and learn more about the speakers and their expertise, without needing to search for information separately.

**3. Advertising and promotional materials during a live event broadcast:** 

It can be used to display advertising and promotional materials, such as sponsored messages or product advertisements. Timed Metadata can also be used to display time-limited offers, discounts, etc, synchronised with the video stream and displayed outside the video, enticing viewers to take advantage of the limited-time deals, creating a sense of urgency, and driving sales.

**4. Timed polls and quizzes for increased engagement**

Polls and quizzes can be used to engage the viewers. Polls can gather opinions, feedback, or preferences from viewers in real-time while quizzes can help test viewer knowledge or understanding of the content being presented. This not only encourages viewer participation but also provides valuable insights to the host or presenter. Using timed metadata for polls and quizzes can help add an element of interactivity and gamification to the live event.

## Conclusion

Geared with ideas to inspire and steps on how to enable Timed Metadata, we are excited to see what you’ll build with it! 🥳

Start by exploring the Timed Metadata docs [here](https://www.100ms.live/docs/javascript/v2/how--to-guides/record-and-live-stream/hls/hls-timed-metadata).

Have any questions? Ask the 100ms team on Discord. Join [here](https://discord.com/invite/kGdmszyzq2).

thumbnail_'.jpg

'.jpg

Timed Metadata for Live Streaming is here!

Akanksha

Live video experiences have become more popular and widely enable use cases such as online meetings, webinars, and virtual events. One of the most common ways to join these live video experiences is through meeting links that allow users to easily access the video platform or application.

Links for live video experiences can be found in calendar invites, WhatsApp messages, emails which redirects to a page to join the live video or even embedded as iframes.

To simplify these experiences, we've launched [Room Codes](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/overview) to quickly launch and go-live with [100ms Prebuilt](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/prebuilt) and custom (self-hosted) room links.

## Room Codes Are Powerful

A 100ms Room can have multiple roles, and each role can be configured with unique and distinct permissions. This lets you define custom experiences for different roles in a 100ms room using [Room Templates](https://www.100ms.live/docs/concepts/v2/concepts/templates-and-roles/overview). 

Once you've set up a 100ms room, your users will need a `[Room link](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/room-links)` to join a room that you’ve created. You'll also need to ensure that your users join a room with the intended role permissions meant for them. [Room Codes](https://www.100ms.live/docs/concepts/v2/concepts/rooms/room-codes/room-codes) allow you to do just that. These are 10 digit unique short codes that represent a room id/role combination.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/room_links_7f50bed2eb/room_links_7f50bed2eb.png) 

Let’s take a look at how [Room Codes](https://www.100ms.live/docs/concepts/v2/concepts/rooms/room-codes/room-codes) can help you integrate 100ms when building 100ms Prebuilt or Custom UI solutions.

## Room codes for Prebuilt links

100ms Prebuilt is a no-code feature rich video conferencing web application, which can be accessed using a [Prebuilt Room Link](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/prebuilt#share-or-embed-as-iframe), or embedded as an iFrame in your application

Generate role specific room links with a format [https://`your-template-subdomain`.app.100ms.live/meeting/`room-code`](https://%60your-template-subdomain%60.app.100ms.live/meeting/%60room-code%60%E2%80%9D), where

- template-subdomain - use 100ms hosted `template-subdomain` when using Prebuilt room links. This is the subdomain that you define at the time of Template creation; can be accessed from Template details page on [100ms Dashboard.](https://dashboard.100ms.live/dashboard)
- room-code -  belongs to a unique `room_id`and `role` for which you're creating the Room Link. Get room codes for each role
    - Copy role specific room code from Join Room section for a role in a room
        
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/ezgif_com_video_to_gif_0e4285c2a7/ezgif_com_video_to_gif_0e4285c2a7.gif) 

 Alternatively, you can use [Create Room Codes API](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api)** to create room codes

### **Steps to generate Prebuilt Links**

1. Create Room Codes using [Room Code APIs](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api) using our [Postman Collection](https://god.gw.postman.com/run-collection/22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a?action=collection%2Ffork&collection-url=entityId%3D22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a%26entityType%3Dcollection%26workspaceId%3Dd9145dd6-337b-4761-81d6-21a30b4147a2)
2. Plug in the room code to the https://<template-subdomain>/meeting/<room-code>

```jsx
# template info

template_id = "639cb27a0c77c9782cc50e7b"
template_subdomain = "<your-template-subdomain>.app.100ms.live"

# create room

room_response = create_room(template_id)

# create room code for the above created room

room_codes = create_room_codes(room_response["id"])

# create create room links by appending room_code to template_subdomain

# room_link = https://<template-subdomain>/meeting/<room-code>

room_links = {}
for data in room_codes:
room_links[data["role"]] = "https://"+template_subdomain+"/meeting/"+data["code"]

print(room_links)
```

### Authenticating peer in Prebuilt

Joining a room requires Auth token. When using Prebuilt, Prebuilt takes care of generating Auth Token implicitly using the room code accessible in the Prebuilt room link. So you don’t have to handle anything!

## Room codes for custom UI solution

When self-hosting your application, build google meet like urls(meet.your.app/abc-pqrs-xyz) and embed those in your custom app, create [Custom Room Links](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/room-links#custom-domain-room-links) hosted on your domain with a format that looks like “[https://`your-domain`/`room-code`](https://%60your-domain%60/meeting/%60room-code%60). 

### Steps to generate custom room links

1. Create a Template, Room and at least one Role on your [100ms account](https://dashboard.100ms.live/dashboard).
2. Use **[Create Room Code](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api)** to generate room codes for one or more roles in a room
3. **[Retrieve Room Codes](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/get-room-code-api)** for each role in the room.
4. Construct your custom domain room link by appending the retrieved **[room code](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/overview)** to the URL format for each role in the room.

- Host Prebuilt Link : https://<your-subdomain>/meeting/`room-code-for-host`
- Guest Prebuilt Link : https://<your-subdomain>/meeting/`room-code-for-guest`

### Room Code Authentication

Room codes can also power join room authentication without setting up an auth token server at your end

1. Generate Room Codes using our [Postman Collection](https://god.gw.postman.com/run-collection/22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a?action=collection%2Ffork&collection-url=entityId%3D22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a%26entityType%3Dcollection%26workspaceId%3Dd9145dd6-337b-4761-81d6-21a30b4147a2); also check [Create Room Code API](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api)
2. Use `getAuthTokenByRoomCode` SDK method to fetch auth token using room code

#### Quick References

- [Room Code Authentication](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/room-code-auth)
- [Quickstart guides](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/room-code-auth#quickstart-with-room-code-authentication) using room code authentication.

### Disabling Room Codes

If your use case doesn’t require joining rooms with a “room link” experience and not sharing links with end users, but rather have an embedded experience built within your application, you may not want to allow peers joining in from room links. In such scenario, you can disable “join with room links” experience by disabling room codes globally for a workspace from [Developer](https://dashboard.100ms.live/developer) section on 100ms dashboard.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/ezgif_com_optimize_0a50e3e9df/ezgif_com_optimize_0a50e3e9df.gif) 

## Conclusion

With Room codes, you can share encrypted ‘short-codes’ in your room link that inherently represents a role and room_id. With unique room codes, roles can join rooms with their unique room links without exposing role or room_id in the link.  

With this, you can get started with room codes and start sharing links with your users. Here are some references to get started

- [Room Codes](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/room-code-overview)
- [Room Code APIs](https://www.100ms.live/docs/server-side/v2/api-reference/room-codes/create-room-code-api)
- [Postman Collection](https://god.gw.postman.com/run-collection/22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a?action=collection%2Ffork&collection-url=entityId%3D22726679-47dcd974-29d5-4965-a35b-bf9b74a8b25a%26entityType%3Dcollection%26workspaceId%3Dd9145dd6-337b-4761-81d6-21a30b4147a2)
- [Room Links](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/room-codes/room-links)
- [100ms Prebuilt](https://www.100ms.live/docs/get-started/v2/get-started/prebuilt/prebuilt)

Have any questions? Ask the 100ms team on Discord. Join [here](https://discord.com/invite/kGdmszyzq2).

thumbnail_Room Links.png

Room Links.png

Create join links with room codes

Pawan Dixit

Ever since the pandemic, online classes have become an integral part of the education system worldwide. We at **100ms** understand the importance of providing a seamless and enjoyable experience for both teachers and students during live teaching classes. We’ll talk about a common problem that teachers face in live classes and the ‘Why, What and How’ of fixing it.

## Autofocus and the Whiteboard

Recently, one of our EdTech customers who use our live SDK reached out to us. There was this certain teacher that worked with them who used a digital whiteboard to teach, and he was facing an issue. Every time he moves in front of the whiteboard, the camera focus swayed with him.

In a traditional **WebRTC** call, the camera uses the default autofocus and exposure mode. This can sometimes lead to the whiteboard becoming out of focus or over/under exposed. And this makes it difficult for students to see what is written on the board clearly.

## Fixing the Focus

We decided to fix this issue by implementing **tap-to-lock-focus** functionality as part of the **100ms iOS SDK**. This allows teachers to easily lock the focus of the camera onto the whiteboard, ensuring that the board remains in focus throughout the class. The **tap-to-focus** functionality is integrated directly into the `HMSVideoView`, which is an easy-to-use API that developers can use to render video during a **WebRTC** call.

The feature includes three different focus modes:

1. `kHMSFocusModeAuto` - The default focus mode, where the camera continuously sets the focus and exposure based on the center point of the scene. This mode is useful when the user wants the camera to automatically adjust to the changing environment or when the user is moving around.
2. `kHMSFocusModeTapToAutoFocus` - This allows the user to tap on an object to focus on it. Once the user moves the camera enough to change the subject of the scene, the camera switches back to the default `kHMSFocusModeAuto`. This mode is useful when the user wants to quickly adjust the focus to a specific object, but still wants the camera to adjust to the changing environment.
3. `kHMSFocusModeTapToLockFocus` - This allows the user to tap on an object to lock focus on it. The focus will not be changed until the user taps again to focus on another object. This mode is useful when the user wants to keep the focus on a specific object, even if the camera is moved.

To use this feature, developers simply had to enable it in the settings—by setting the camera focus mode to one of the modes mentioned above. We made it easy to implement and roll out, allowing teachers to start using it as soon as possible. Now, let’s have a look at how we implemented it!

## Behind the Scenes

The class that does the capturing from device camera on **WebRTC** is `RTCCameraVideoCapturer`. This gives us an array of camera devices available. We can use either the front or the back camera to capture the local video. This capture device is an `AVCaptureDevice` that represents a physical camera on iPhone.

Implementing **tap-to-focus** had mainly two parts to it:

1. Enabling the user to be able to tap on the local video tile to let the SDK know what part of the video should get focused.
2. Sending the Focus information to the video capturer.

**100ms SDK** provides a simple interface to render video called `HMSVideoView`. We installed a tap handler on `HMSVideoView` such that if the view is being used to render local video, it calculates the fractional coordinates (range 0-1.0) of the tap location inside the view. Note that fractional coordinates was essential because then you don't have to deal with scaling of the video layer itself. Another thing to keep in mind is that we need to calculate the tapping point taking into account the orientation of the device.

```swift
UIInterfaceOrientation currentOrienation;
    
    if (@available(iOS 13.0, *)) {
        currentOrienation = UIApplication.sharedApplication.windows.firstObject.windowScene.interfaceOrientation;
    } else {
        // Fallback on earlier versions
        currentOrienation = UIApplication.sharedApplication.statusBarOrientation;
    }
    
    CGPoint focusPoint;
    
    if (currentOrienation == UIInterfaceOrientationLandscapeLeft || currentOrienation == UIInterfaceOrientationLandscapeRight) {
        focusPoint = CGPointMake([touchPoint locationInView:self].x / viewSize.width, [touchPoint locationInView:self].y / viewSize.height);
    }
    else {
        focusPoint = CGPointMake([touchPoint locationInView:self].y / viewSize.height, 1.0 - [touchPoint locationInView:self].x / viewSize.width);
    }
```

Now comes the second part. We send these fractional coordinates to the video capturer. Here we set the `AVCaptureDevice`'s focal point and exposure according to the focused point on screen.

```swift
[self.device lockForConfiguration:nil];
    
if (self.device.isFocusPointOfInterestSupported) {
	self.device.focusPointOfInterest = point;
	self.device.focusMode = AVCaptureFocusModeAutoFocus;
}
    
if (self.device.isExposurePointOfInterestSupported) {
	self.device.exposurePointOfInterest = point;
	self.device.exposureMode = AVCaptureExposureModeAutoExpose;
}
    
[self.device unlockForConfiguration];
```

There is one more thing remaining. If you see the **tap-to-focus** feature on the native iOS camera app, you can notice that the camera switches back to auto focus when you move the camera too much from the current scene. This is the same functionality we wanted to offer in our `TapToAutoFocus` mode. 

To accomplish this we have to know exactly ‘when’ the scene has changed enough, so that we can reset the focus and exposure to auto. To do this, we observe `AVCaptureDeviceSubjectAreaDidChange` notification from iOS notification center, and reset to autofocus.

```swift
subjectAreaDidChangeObserver = [NSNotificationCenter.defaultCenter addObserverForName:AVCaptureDeviceSubjectAreaDidChangeNotification object:nil queue:NSOperationQueue.mainQueue usingBlock:^(NSNotification * _Nonnull note) {
	[self resetFocusAndExposure];
}];
```

## The End

This feature ensures that the whiteboard remains in focus during the class, making it easy for students to see what is written on the board and follow along with the class. And finally, the teacher was happy with the **tap-to-lock-focus** implemented in the **SDK** and so was our customer. Thanks for reading!

At **100ms**, we emphasize developer-friendly APIs that make it easy for our customers to implement new features. The **tap-to-focus** is no exception, and we have made it as simple as possible to use. To learn more about this feature, please visit our developer documentation at [Tap-to-focus](https://www.100ms.live/docs/ios/v2/features/tap-to-focus).

large_Thumbnail (23).png

medium_Thumbnail (23).png

small_Thumbnail (23).png

thumbnail_Thumbnail (23).png

Thumbnail (23).png

Enhancing Live Teaching Experience with Tap-to-Focus in 100ms SDK

If you’re someone who has just start building with 100ms live video SDKs, you might have noticed that there are certain things you can’t do with just the SDK. From creating a new room, to generating an Auth Token to join a room, the Dashboard is your only friend—or so you thought.

Any application built using 100ms SDK consists of 2 components:

1. **Client** - The frontend app that uses the 100ms SDK (Android, iOS, Web, React Native, Flutter) for the end-users.
2. **Server** - The backend that uses 100ms REST APIs to create rooms, trigger recording or Live streaming, access webhook events.

Everything you can do with the 100ms REST APIs (Server) can be done in your Dashboard. But, the Dashboard being your only option is not recommended in production. Before we start, here are some 100ms terminologies you should be familiar with:

1. **Room** - A room is a virtual space where one or more peers communicate with each other. Rooms map to real-world entities: a classroom, tele-health consultation, or a webinar. You can imagine a physical room in a building that is open at all times, where anyone authorized can enter and leave.
2. **Auth Token** - Used to authenticate and allow end-users (peers) to join 100ms rooms. This can be considered an entry pass that authorizes a peer to enter a specific room.
3. **Session** - Session is a single continuous call in a room. A session begins when the first peer joins an empty room and ends when the last peer leaves it. A single room can have multiple sessions.

In this guide, we’ll build a sample backend app in Node.js that uses 100ms REST APIs to do the following:

1. Create a new Room
2. Generate Auth Token for joining a Room
3. Get usage analytics for the latest Session in the Room

Now that we know what we’ll be building, let’s begin!

## Initialize Node.js app

Let us start by creating a new project directory `100ms-sample-token-server-nodejs`. Then, the following command is used to create `package.json` with configurations needed for a Node.js App.

```bash
npm init
```

On executing the command, when prompted for Entry point, type `src/app.js`.

## Setting up Express with Node.js

We’ll be using Express.js for serving the HTTP endpoints. To use the Express framework, we need to install it as a dependency first.

```bash
npm i --save express
```

Now create a file called `app.js` inside a new directory called `src` and create an express app inside it by using the following code.

```js
let express = require('express');
let app = express();
app.use(express.json()); // for parsing request body as JSON
let port = process.env.PORT || 3000;

app.listen(port, () => {
    console.log(`Token server started on ${port}!`);
});
```

## Installing Additional Dependencies

Install the following dependencies,

```bash
npm i axios dotenv jsonwebtoken moment uuid4
```

Here’s what we’ll be using these dependencies for:

1. `axios` → for making API calls to 100ms servers
2. `dotenv` → for loading environmental variables from `.env` file
3. `jsonwebtoken` → for generating app and management tokens (JWT tokens) 
4. `uuid4` → for generating random UUID
5. `moment` → for time related operations

## Configuring Environment Variables

Now, login to your 100ms Dashboard and go to the Developer section. You will find 2 values that you’d require for both generating the Auth Token and making 100ms REST API calls.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Untitled_6fb7d1353a/Untitled_6fb7d1353a.png) 

Now, copy the values of “App Access Key” and “App Secret” into a file called `.env`.

```
APP_ACCESS_KEY=<YOUR_APP_ACCESS_KEY>
APP_SECRET=<YOUR_APP_SECRET>
```

## Setting up Token Service

We will be dealing with two kinds of tokens here: 

1. **Auth Token** - a token that would enable a client to join a room.
2. **Management Token** - a token that would enable the app(this server) to communicate with 100ms REST APIs.

To generate and manage these tokens, we will now create a service class `TokenService` inside the `src/services/TokenService.js` file.

In order to generate a token, we will sign the payload that contains the App Access Key along with several other configurations, using the JWT mechanism with the App Secret as key. First, let us create 2 private methods—one to sign the payload using JWT and the other to check if an existing token has expired.

>If you’re a TypeScript person, the symbol `#` is [used to make a class member private](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_class_fields) in JavaScript. It is the equivalent of using the keyword `private` in TS.


```js
let jwt = require('jsonwebtoken');
let uuid4 = require('uuid4');

// A service class for Token generation and management
class TokenService {
    static #app_access_key = process.env.APP_ACCESS_KEY;
    static #app_secret = process.env.APP_SECRET;
    #managementToken;
    constructor() {
        this.#managementToken = this.getManagementToken(true);
    }

    // A private method that uses JWT to sign the payload with APP_SECRET
    #signPayloadToToken(payload) {
        let token = jwt.sign(
            payload,
            TokenService.#app_secret,
            {
                algorithm: 'HS256',
                expiresIn: '24h',
                jwtid: uuid4()
            }
        );
        return token;
    }

    // A private method to check if a JWT token has expired or going to expire soon
    #isTokenExpired(token) {
        try {
            const { exp } = jwt.decode(token);
            const buffer = 30; // generate new if it's going to expire soon
            const currTimeSeconds = Math.floor(Date.now() / 1000);
            return !exp || exp + buffer < currTimeSeconds;
        } catch (err) {
            console.log("error in decoding token", err);
            return true;
        }
    }
}

module.exports = {TokenService};
```

Now, in order to generate these tokens, we’ll add the following the methods:

1. `getManagementToken()` - Checks if the existing Management Token has expired. Generates a new token in that case and returns it.
2. `getAuthToken()` - Generates a new Auth Token using the room id, user id and role and returns it.

```js
class TokenService {
...
		// Generate new Management token, if expired or forced
	  getManagementToken(forceNew) {
        if (forceNew || this.#isTokenExpired(this.#managementToken)) {
            let payload = {
                access_key: TokenService.#app_access_key,
                type: 'management',
                version: 2,
                iat: Math.floor(Date.now() / 1000),
                nbf: Math.floor(Date.now() / 1000)
            };
            this.#managementToken = this.#signPayloadToToken(payload);
        }
        return this.#managementToken;
    }

    // Generate new Auth token for a peer
    getAuthToken({ room_id, user_id, role }) {
        let payload = {
            access_key: TokenService.#app_access_key,
            room_id: room_id,
            user_id: user_id,
            role: role,
            type: 'app',
            version: 2,
            iat: Math.floor(Date.now() / 1000),
            nbf: Math.floor(Date.now() / 1000)
        };
        return this.#signPayloadToToken(payload);
    }
}
```

## Setting up API Service

To make REST API calls to 100ms servers from this app, we’ll use a HTTP client called `axios`. We will create a service class `APIService` inside the `src/services/APIService.js` to handle all the REST api calls with a configured `axios` instance.

We will use the `constructor` and a private method `configureAxios()` to create and configure an `axios` instance:

- With the baseURL set to `"https://api.100ms.live/v2"`
- Uses the Management token from the `TokenService` for Bearer Authorization
- Retries a failed request with a newly generated Management token

```js
const axios = require('axios').default;

// A service class for all REST API operations
class APIService {
    #axiosInstance;
    #tokenServiceInstance
    constructor(tokenService) {
        // Set Axios baseURL to 100ms API BaseURI
        this.#axiosInstance = axios.create(
            {
                baseURL: "https://api.100ms.live/v2",
                timeout: 3 * 60000
            });
        this.#tokenServiceInstance = tokenService;
        this.#configureAxios();
    }

    // Add Axios interceptors to process all requests and responses
    #configureAxios() {
        this.#axiosInstance.interceptors.request.use((config) => {
            // Add Authorization on every request made using the Management token
            config.headers = {
                Authorization: `Bearer ${this.#tokenServiceInstance.getManagementToken()}`,
                Accept: "application/json",
                "Content-Type": "application/json",
            };
            return config;
        },
            (error) => Promise.reject(error));
        this.#axiosInstance.interceptors.response.use((response) => {
            return response;
        },
            (error) => {
                console.error("Error in making API call", { response: error.response?.data });
                const originalRequest = error.config;
                if (
                    (error.response?.status === 403 || error.response?.status === 401) &&
                    !originalRequest._retry
                ) {
                    console.log("Retrying request with refreshed token");
                    originalRequest._retry = true;

                    // Force refresh Management token on error making API call
                    this.axios.defaults.headers.common["Authorization"] = "Bearer " + this.#tokenServiceInstance.getManagementToken(true);
                    try {
                        return this.axios(originalRequest);
                    } catch (error) {
                        console.error("Unable to Retry!");
                    }
                }
                return Promise.reject(error);
            }
        );
    }
}

module.exports = {APIService};
```

We will also add the methods `get()` and `post()` that uses this configured `axios` instance to make requests and return the data from the response. These are the methods we will use to make the 100ms REST API calls.

```js
class APIService {
...
		// A method for GET requests using the configured Axios instance
    async get(path, queryParams) {
        const res = await this.#axiosInstance.get(path, { params: queryParams });
        console.log(`get call to path - ${path}, status code - ${res.status}`);
        return res.data;
    }

    // A method for POST requests using the configured Axios instance
    async post(path, payload) {
        const res = await this.#axiosInstance.post(path, payload || {});
        console.log(`post call to path - ${path}, status code - ${res.status}`);
        return res.data;
    }
}
```

## Create a New Room

Now, create a POST request handler for the route `/create-room` to create a new Room using the 100ms REST APIs. And the arguments for [creating a room](https://www.100ms.live/docs/server-side/v2/Rooms/create-via-api), like name and description of the room are specified in the body of the request.

The client can use this endpoint to create a new room as and when needed. It is the responsibility of the client to store the generated room id, to use it in the subsequent requests to generate Auth Token and the usage analytics.

```js
// Create a new room, either randomly or with the requested configuration
app.post('/create-room', async (req, res) => {
    const payload = {
        "name": req.body.name,
        "description": req.body.description,
        "template_id": req.body.template_id,
        "region": req.body.region
    };

    try {
        const resData = await apiService.post("/rooms", payload);
        res.json(resData);
    } catch (err) {
        console.error(err);
        res.status(500).send("Internal Server Error");
    }
});
```

## Serving the Auth Token

Now back to the `src/app.js`, create an instance of both `TokenService` and `APIService` to use in the request handlers.

Create a POST request handler for the route `/auth-token` to serve the Auth Token to the clients. Use the `getAuthToken()` method from the `TokenService` instance to generate one and send it as response.

The client can use the room id of the newly generated room and make this request. Once the Auth Token has been generated, the client can use it to join that room using the 100ms SDK.

```js
...

require('dotenv').config();
const moment = require('moment');
const { APIService } = require('./services/APIService');
const { TokenService } = require('./services/TokenService');

const tokenService = new TokenService();
const apiService = new APIService(tokenService);

// Generate an auth token for a peer to join a room
app.post('/auth-token', (req, res) => {
    try {
        const token = tokenService.getAuthToken({ room_id: req.body['room_id'], user_id: req.body['user_id'], role: req.body['role'] });
        res.json({
            token: token,
            msg: "Token generated successfully!",
            success: true,
        });
    } catch (error) {
        res.json({
            msg: "Some error occured!",
            success: false,
        });
    }
});
```

## Get usage analytics for the latest Session in the Room

Finally, create a GET request handler for the route `/session-analytics` that does the following:

1. Fetch all the sessions for a specific room.
2. Processes the session details of the latest session among them, to generate usage analytics.

Here, we take the [example of Attendance](https://www.100ms.live/docs/server-side/v2/Sessions/example-build-attendance). The session details are fetched from the [API call based on room id](https://www.100ms.live/docs/server-side/v2/api-reference/Sessions/list-sessions) and then the response is used to calculate the following:

- Individual participants' duration in the call
- Aggregated participants' duration
- Total session duration

```js
// Get usage analytics for the latest Session in a Room
app.get('/session-analytics-by-room', async (req, res) => {
    try {
        const sessionListData = await apiService.get("/sessions", { room_id: req.query.room_id });
        if (sessionListData.data.length > 0) {
            const sessionData = sessionListData.data[0];
            console.log(sessionData);

            // Calculate individual participants' duration
            const peers = Object.values(sessionData.peers);
            const detailsByUser = peers.reduce((acc, peer) => {
                const duration = moment
                    .duration(moment(peer.left_at).diff(moment(peer.joined_at)))
                    .asMinutes();
                const roundedDuration = Math.round(duration * 100) / 100;
                acc[peer.user_id] = {
                    "name": peer.name,
                    "user_id": peer.user_id,
                    "duration": (acc[peer.user_id] || 0) + roundedDuration
                };
                return acc;
            }, {});
            const result = Object.values(detailsByUser);
            console.log(result);

            // Calculate aggregated participants' duration
            const totalDuration = result
                .reduce((a, b) => a + b.duration, 0)
                .toFixed(2);
            console.log(`Total duration for all peers: ${totalDuration} minutes`);

            // Calculate total session duration
            const sessionDuration = moment
                .duration(moment(sessionData.updated_at).diff(moment(sessionData.created_at)))
                .asMinutes()
                .toFixed(2);
            console.log(`Session duration is: ${sessionDuration} minutes`);

            res.json({
                "user_duration_list": result,
                "session_duration": sessionDuration,
                "total_peer_duration": totalDuration
            });
        } else {
            res.status(404).send("No session found for this room");
        }
    } catch (err) {
        console.error(err);
        res.status(500).send("Internal Server Error");
    }
});
```

## Deploy on Render

If you want to try it out without much trouble, you can deploy this app on Render for free. Make sure to specify the environment variables when asked.

[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/100mslive/100ms-sample-backend-nodejs)

You can check out the complete source code here at [100ms-sample-backend-nodejs](https://github.com/100mslive/100ms-sample-backend-nodejs). Don’t forget to star the repo if it was useful to you!

large_NodeJS - 1 (1).png

medium_NodeJS - 1 (1).png

small_NodeJS - 1 (1).png

thumbnail_NodeJS - 1 (1).png

NodeJS - 1 (1).png

Building a sample backend server for your 100ms app with Node.js

Arjun Attam

100ms started out as an SDK for real-time video conferencing, with 10,000 free minutes of usage for developers to try out the product. Since then, we’ve added new capabilities: [interactive live streaming](https://www.100ms.live/docs/javascript/v2/foundation/live-streaming), external streaming to social media platforms, and various [forms of recording](https://www.100ms.live/docs/javascript/v2/foundation/recordings).

Today, we’re updating [our pricing model](https://www.100ms.live/pricing) with free minutes of usage for all capabilities. This ensures developers can sign up and try out *all* of 100ms, with sufficient free minutes available for product evaluation. This includes the following free minutes:

- Real-time video conferencing: 10,000 mins
- Interactive live streaming: 10,000 viewer minutes and 1,000 encoding minutes
- External streaming: 300 mins
- Recording: 300 mins

These changes will be applicable from the January 2023 billing cycle, and will reflect on your upcoming invoices (if you’ve crossed the free limit).

In addition to the increased free minutes, we’re shipping new features to the 100ms Dashboard to make your billing experience more self-serve.

## Complete usage overview

Usage overview on the 100ms Dashboard has been updated to include all capabilities, along with a summary of total used minutes and remaining free minutes. This gives you more observability on your usage patterns, and helps you predict an upcoming invoice.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2023_02_02_at_3_15_52_PM_95364c9d19/Screenshot_2023_02_02_at_3_15_52_PM_95364c9d19.png) 

## Review and pay invoices

We’ve shipped an invoices page for you to track new and past invoices. Outstanding invoices can be paid directly from here.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2023_02_02_at_2_03_40_PM_89cc8b7c21/Screenshot_2023_02_02_at_2_03_40_PM_89cc8b7c21.png) 

## Add payment method

The 100ms Dashboard will also allow you to configure a payment method on your organisation account. We use Stripe to securely store your credit card information. This feature will be available in the second week of February – in time for your next billing invoice.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2023_02_02_at_2_54_31_PM_b7feb7d9c5/Screenshot_2023_02_02_at_2_54_31_PM_b7feb7d9c5.png) 

That’s all for now! If you have any feedback, hit us up in the chat widget on the dashboard or on [Discord](https://100ms.live/discord). We hope this extension of free minutes and self-serve billing improves your experience of building with 100ms.

large_Updates to Billing-2-min.png

medium_Updates to Billing-2-min.png

small_Updates to Billing-2-min.png

thumbnail_Updates to Billing-2-min.png

Updates to Billing-2-min.png

New free tiers and self-serve billing

Pratim Mallick

One of our customers is a social audio rooms app on Android where hosts go live on talk shows. Recently, they came up with a feature request. They wanted to convert a few of their rooms into "radio stations", where the host of the room would stream local music from their device, and also talk simultaneously, so that listeners would be able hear the voice of host as well as the music together.

This meant that we needed the ability to mix data captured from the device’s microphone and the music captured from the local device. This was a challenging feature to be built using native WebRTC on mobile devices, since unlike WebRTC on web, WebRTC does not expose any API to add custom audio tracks.

To fulfil this scenario, we made significant changes to WebRTC and our Android SDK. We will go over those changes in this post — starting with an overview of how audio is captured by WebRTC, the changes we made to the WebRTC source, and how to mix two audio streams programmatically.

To make things easier to explain, the problem statement is broken down into 3 sub problems 

1. How to capture audio from the mic of the device 
2. How to capture local audio from the device
3. How to mix two audio streams 

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Local_Audio_Share_8f2b07e1dc/Local_Audio_Share_8f2b07e1dc.jpg) 
 
 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Screenshot_2023_02_03_at_7_13_54_PM_b23f8f90c2/Screenshot_2023_02_03_at_7_13_54_PM_b23f8f90c2.png) 

Let’s go step by step into solving each of these problems. 

## 1. How to capture audio from the mic of the device 

To find a solution to this we would first need to understand the flow inside of WebRTC SDK that captures audio from the mic of a device. 

So let’s dive in!

Things start off when we create an instance of `PeerConnectionFactory` of WebRTC. It internally creates an instance of `JavaAudioDeviceModule`, which is an implementation of WebRTC’s `AudioDeviceModule`. 

### JavaAudioDeviceModule

The `JavaAudioDeviceModule` is the main class responsible for setting parameters while capturing audio(*Input*) or playing out audio(*Output*). Some of the things that this class is responsible for are :

- Setting preferred input device
- Enabling/Disabling noise suppression
- Input/Output sampling rate
- Input/Output Stereo/mono channel

It also creates instances of  `WebrtcAudioRecord` (for Input) and `WebrtcAudioTrack` (for output) which are the actual classes responsible for capturing audio from the mic and playing out audio.

Let’s look deeper into `WebRtcAudioRecord` since that is where lies the path to our solution.

### WebrtcAudioRecord

This class uses Android’s `AudioRecord` to communicate with the device to open and capture an audio stream from the mic. 

When WebRTC starts to record an audio stream from the system’s mic, a `Java Thread` is spawned whose main responsibility is as follows:

1. Read audio buffer from the input hardware for recording into a direct `buffer` continuously 
2. Send the number of bytes reads and the buffer containing actual audio bytes to the native layer of WebRTC, so that it can be encoded, packetised and streamed over network 

Now that we have looked at the main classes of WebRTC that come into play while capturing audio from a device, we need a way for us to be able to get these bytes so that we can modify them before they are sent over the network. 

To achieve this we needed to make changes in the WebRTC’s JAVA layer. Following are the changes that were required :

1. Create a callback  that would be called as soon as data is read from device's microphone and it would contain the `Byte buffer` that was read.

```kotlin
public interface AudioBufferCallback {
        void onBuffer(ByteBuffer buffer, int bytesRead);
    }
```

2. Modify `JavaAudioDeviceModule` 's creation logic, so that we can provide the above callback instance when we create  `PeerConnectionFactory`

```kotlin
// Create webrtcAudioRecord
webrtcAudioRecord = WebRtcAudioRecord(
		...
    )

// Set the callback to webrtcAudioRecord
webrtcAudioRecord.setBufferCallback(
      object : AudioBufferCallback {
    override fun onBuffer(micByteBuffer: ByteBuffer?,  micBytesRead: Int) {
      
        }
    )
})

// Create instance of JavaAudioDeviceModule & set webrtcAudioRecord to it
val customAudioDeviceModule = JavaAudioDeviceModule(
      ..
      webrtcAudioRecord,
      ..
    )

// set the above JavaAudioDeviceModule instance while creating PeerConnectionFactory instance
val peerConnectionFactory = PeerConnectionFactory.builder()
			..
      .setAudioDeviceModule(customAudioDeviceModule)
      .createPeerConnectionFactory()
```

With these changes, we now have the captured audio in the form of byte buffer with us whenever `onBuffer` gets called. 

Now let’s move on to the next problem

## 2. How to capture local audio from the device

So now that we have the bytes captured from device's mic, our next goal is to get the audio from the local device so that we can mix them together

To capture system audio we use Android’s `AudioRecord` class to record audio played by other apps on the device and that allows their audio output to be captured. **This method is available only on Android 10 and above. **

To use the Audio Record object to capture other app’s audio, we first create an instance of AudioRecord and set its format and playback config

```kotlin
// Sets the format of the audio data to be captured.
val format = AudioFormat.Builder()
      .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
      .setSampleRate(48000) // Most devices operate at 48kHz sampling rate
      .setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
      .build()

// Sets the audioRecord to record audio played by other apps
val playbackConfig = AudioPlaybackCaptureConfiguration.Builder(mMediaProjection)
      .addMatchingUsage(AudioAttributes.USAGE_MEDIA)
      .addMatchingUsage(AudioAttributes.USAGE_UNKNOWN)
      .addMatchingUsage(AudioAttributes.USAGE_GAME)
      .build()

// Create the audio Record instance 
val mAudioRecord = AudioRecord.Builder()
      .setAudioFormat(format)
      .setAudioPlaybackCaptureConfig(playbackConfig)
      .build()

// Starts recording from the AudioRecord instance
mAudioRecord.startRecording()
```

Once the audio record is created, the next step is to read the audio from other apps in the system. But the question is when and how much to read at a time. 

This is where the callback we created in the first step comes into the picture

The `onBuffer` callback of `AudioBufferCallback` interface is called whenever webrtc’s audio capture thread reads the audio bytes from the microphone. This callback gives us two values. 

1. `micByteBuffer` - which is the ByteBuffer that contains the bytes read by WebRTC from the microphone of the device
2. `micBytesRead`  - An Integer value that gives us how many bytes were read by WebRTC from the mic of the device

So now we know when to read the audio of other apps using the instance of `audioRecord` created above - whenever we receive the callback. And we also know how much to read - the value of the parameter `micBytesRead` received in the callback. 

We read from the audio record into a ByteBuffer like so:

```kotlin
// Allocates a new direct byte buffer.
val localAudioByteBuffer: ByteBuffer = ByteBuffer.allocateDirect(micBytesRead).order(ByteOrder.nativeOrder())
// Reads audio data from the audio record for recording into a direct buffer
mAudioRecord.read(localAudioByteBuffer, micBytesRead, READ_BLOCKING)
```

This would give us the audio bytes from other apps/systems into the newly created `localAudioByteBuffer`. 

## 3. How to mix two audio streams 
Now that we have captured audio from both sources - local and mic - all we need is to mix them to create a combined stream that can be streamed to other peers connected via WebRTC.

The way to create a new buffer is to first convert the `ByteBuffer` into `ShortArray`. 

```kotlin
// a1 and a2 are the ShortArrays that needs to be combined
// a1Limit and a2Limit are the sizes of the above arrays  
fun addBuffers(
        a1: ShortArray,
        a1Limit: Int,
        a2: ShortArray,
        a2Limit: Int
    ): ByteArray {
        val size = Math.max(a1Limit, a2Limit)
        if (size < 0) return ByteArray(0)
        val result = ByteArray(size * 2)
        for (i in 0 until size) {
            var sum: Int
            sum = if (i >= a1Limit) {
                a2[i].toInt()
            } else if (i >= a2Limit) {
                a1[i].toInt()
            } else {
                a1[i].toInt() + a2[i].toInt()
            }
            if (sum > Short.MAX_VALUE) sum = Short.MAX_VALUE.toInt()
            if (sum < Short.MIN_VALUE) sum = Short.MIN_VALUE.toInt()
            val byteIndex = i * 2
            result[byteIndex] = (sum and 0xff).toByte()
            result[byteIndex + 1] = (sum shr 8 and 0xff).toByte()
        }
        return result
    }
```

 The above method will return you a `ByteArray` which needs to be converted back to `ByteBuffer` before passing on to WebRTC.

Once the mixing of the audio is done, all we need to do is clear the contents of the microphone `ByteBuffer` captured by WebRTC and replace it with the resultant Byte buffers from the above  

```kotlin
// Clear the mic buffer
micAudioByteBuffer.clear()
micAudioByteBuffer.put(combinedByteBuffer)
```

This will lead to streaming the combined audio from the microphone as well as the one playing on local device as a single audio stream. We can also replace mic audio with local device audio - if there is a use case for that. 

Please note that since this involves using MediaProjection APIs of Android to capture the system audio, this would only work on Android 10 and above. 

large_audioapps-with100ms

medium_audioapps-with100ms

small_audioapps-with100ms

thumbnail_audioapps-with100ms

audioapps-with100ms

How we built Local Audio Sharing in Android

Here's what we shipped this month - 

## THE BIG THING

 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Group_10171_1_3fa746326c/Group_10171_1_3fa746326c.png) 

### [Teams & Workspaces](https://www.100ms.live/blog/launching-teams-workspaces)
Today we launch one of our highly requested features: Teams and Workspaces. You can now create workspaces on the 100ms Dashboard, and invite team members to them. With this, collaboration becomes easier for a range of use cases: for instance, your team doesn’t have to juggle between different accounts for your production and development environments. Set it up with the following steps, and you're good to go:

- Create distinct deployment environments on the dashboard by creating a new workspace called “Prod”. Change your existing workspace name from “Default” to “Dev”
- Invite members to their respective workspaces
- These members will be able to access your workspace and do things like create/modify templates, create rooms, see room data, etc
- Add distinct production and development webhook URLs to each workspace to listen to workspace-specific events


 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Teams_and_Workspaces_Screenshot_1_29aad7e6ff/Teams_and_Workspaces_Screenshot_1_29aad7e6ff.png) 

***You can also maintain separate workspaces in case you’re running multiple use-cases on 100ms (e.g - livestreaming and video calls). [This blog](https://www.100ms.live/blog/launching-teams-workspaces) covers other interesting, cool things you can do with workspaces, which I think you should read right away!***

## ANOTHER BIG THING

 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Group_10170_3_03ae74d21c/Group_10170_3_03ae74d21c.png) 

### [Adaptive Bitrate(aka Simulcast)](https://www.100ms.live/docs/javascript/v2/foundation/adaptive-bitrate)
One of our core ethos from Day 0 is "video should always work", and we're excited to announce a huge upgrade to uphold this promise. Video calls on 100ms now support adaptive bitrate, which means participants can dynamically subscribe to lower/higher video quality of their peers based on available bandwidth and the size of the video element.

This is made possible by participants "simulcasting", or publishing multiple quality layers of their video (for example, in 180p, 360p, 720p).

You can enable this feature in your template configuration in the dashboard:

 ![](https://storage.googleapis.com/100ms-cms-dev/cms/image_6_73e63304d9/image_6_73e63304d9.png) 

***More about how to enable this can be found in the [docs](https://www.100ms.live/docs/javascript/v2/foundation/adaptive-bitrate)***


## The small things

- **Tap and lock camera focus on iOS** - Super useful in virtual classrooms when the teacher wants to lock the focus of camera
 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Tap_and_lock_1b44ef6e41/Tap_and_lock_1b44ef6e41.gif) 

- **Pinch and zoom on mobile SDKs** - Enables participants to pinch and zoom when peers are sharing a document via screen-share
 ![](https://storage.googleapis.com/100ms-cms-dev/cms/output_onlinegiftools_4b2f6bbefe/output_onlinegiftools_4b2f6bbefe.gif) 

- **Billing comes to your dashboard** - Accessing billing information and invoice payments is now easy and simple!
 ![](https://storage.googleapis.com/100ms-cms-dev/cms/billing_a5daf02a5f/billing_a5daf02a5f.png) 
 
- **Virtual background plugin is updated** for better quality with higher frame rates and reduced CPU usage.

- **Picture-in-Picture support is now extended** to [React Native](https://www.100ms.live/docs/react-native/v2/features/pip-mode) and [Flutter](https://www.100ms.live/docs/flutter/v2/advanced-features/pip-mode).
 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Frame_1171274949_3_dcb74a1101/Frame_1171274949_3_dcb74a1101.webp) 

## Other happen-things

Online learning is a super interesting space, even if you’re not into it. People trying to learn from each other is pretty much how the internet was born, so how it continues to evolve is always worth understanding.

100ms hosted a roundtable last week, with 3 product leaders who are building the new wave of community-led learning. Check out the session recording featuring leaders from Maven, Kajabi and Nas Company.

 ![](https://storage.googleapis.com/100ms-cms-dev/cms/Frame_1338_ff4d8d91b2/Frame_1338_ff4d8d91b2.png) 


----------


Here are the latest release notes for all our client-side SDKs, so you’re all caught up. Please upgrade to the latest version to experience the best of 100ms. 👇 


[JavaScript(v0.8.1)](https://www.100ms.live/docs/javascript/v2/changelog/release-notes) | [ReactJS(v0.6.1)](https://www.100ms.live/docs/javascript/v2/changelog/release-notes)

[React Native(v1.1.0)](https://www.100ms.live/docs/react-native/v2/changelog/release-notes) | [iOS(v0.5.3)](https://www.100ms.live/docs/ios/v2/changelog/release-notes) | [Android (v2.5.5)](https://www.100ms.live/docs/android/v2/changelog/release-notes) | [Flutter(v1.1.0)](https://www.100ms.live/docs/flutter/v2/changelog/release-notes)


----------

That's it from us. If you've read this far, thank you so, so much 🙂 it is people like you that keep us going. On behalf of team 100ms, I want to wish you a very happy new year. Here it is:

**A VERY HAPPY NEW YEAR! 🎊**

Cheers,
Rishabh


large_RBAC2 (2).png

medium_RBAC2 (2).png

small_RBAC2 (2).png

thumbnail_RBAC2 (2).png

RBAC2 (2).png

December 2022 Updates: Workspaces, Adaptive Bitrate, interactivity gestures✋ and more

Nitin Ranganath

Damilare Jolayemi

Kshitij Gupta

Rishabh Agnihotri

Aniket Kadam

Shreya Bose

Govind Maheshwari

Developers use 100ms SDKs to build diverse live video features in their apps. Imagine features that enable a virtual classroom, a telehealth consultation, or virtual visits to real estate locations. Unlike standalone apps like Zoom or Google Meet, features built with 100ms are deeply integrated within an app. This integration is possible through a collaboration of multiple developers in a team: web or mobile developers build feature UI, and backend developers set up authentication with 100ms tokens.

Features built with 100ms need to work in different deployment environments, with a clean separation between development and production so that experimentation during development does not affect production.

Until yesterday, the only way to achieve this setup was to create different accounts on 100ms and share access with your colleagues—*not ideal,* to say the least. To fix this and enable these scenarios better, the [100ms dashboard](https://dashboard.100ms.live/dashboard) now supports “workspaces” that enable isolated deployment environments and team collaboration within them. Head over to your dashboard account to try it out now!

This was a highly requested feature by many of you, and we appreciate your patience and feedback that helped us ship this. Let’s take a deeper look at workspaces in this blog post.

## Workspaces

A workspace is an isolated environment which acts as a container for your templates and rooms. Usage and activity data of templates and rooms are stored in a workspace. You can **create or switch to a workspace** using the workspace switcher on the top-left section of your dashboard. These workspaces belong to a single organization account.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/SS_Switcher_2_92115dc3bb/SS_Switcher_2_92115dc3bb.png) 

**Workspace-level Tokens and Webhooks**

Each workspace has a set of respective `access key` and `secret` for [generation of tokens](https://www.100ms.live/docs/javascript/v2/foundation/security-and-tokens). This separation of tokens per workspace enables you to control access for different environments.

Just like tokens, each workspace has its own webhook configuration. This enables you to define different webhook endpoints for each workspace and distinguish events from different deployment environments. The tokens and webhook configuration for a workspace are available in the [developer section](https://dashboard.100ms.live/developer) of your dashboard.

To understand workspaces better, let’s take an example use case: **creating development and production environments**. Create a workspace for development and another workspace for production. Your access key, secret and webhook configuration will be distinct for these two environments.

As another example, you can also create separate workspaces for different product offerings. Let’s say you have two product offerings: a small classroom which runs on video conferencing and a webinar classroom which runs on live streaming. You can create workspaces for each of these products and add members of those teams into the respective workspaces. 

Take your first step in exploring this feature by creating a new workspace or changing the name of your default workspace in [workspace settings](https://dashboard.100ms.live/settings/workspace). 

## Members in workspaces

You can now **add members** to each of your workspaces to create workspace-specific teams. Members are specific to a workspace and won’t show up in other workspaces unless explicitly added.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/Members_Screenshot_Final_023f06d9d5/Members_Screenshot_Final_023f06d9d5.png) 

Members who belong to a workspace will have the ability to edit templates, create rooms, and modify settings based on their membership type. The following table outlines available membership types and their respective access permissions.

| Member Type | Rooms (Create or modify) | Templates (Create or modify) | Developer Configuration | Workspace Settings | Organization Settings  | Billing |
| --- | --- | --- | --- | --- | --- | --- |
| Organization Owner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Workspace Admin (coming soon) | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Workspace Member  | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |

Get started by [inviting members](https://dashboard.100ms.live/settings/workspace) to your workspace.

## What’s coming up next

With today’s release, workspaces are available for you to try out. Here’s a sneak peek at what’s coming in subsequent releases.

- Workspace Admin as a new member type for more granular access control
- Ability to clone templates across workspaces, so that you can upgrade template config changes to production
- Ability to delete workspaces
- Auto-add members to a workspace for the same work domain
- Migration support for existing multiple accounts into workspaces of a single organization account

Do you have a preferred feature from the list above? Are we missing something? Want to try out an early preview of one of these features? Drop us a line at *hello@100ms.live*

Launching Teams and Workspaces

Vrushti Oza

large_livestream with 100ms-V2.png

medium_livestream with 100ms-V2.png

small_livestream with 100ms-V2.png

thumbnail_livestream with 100ms-V2.png

local-audio-share-android.png

Test for multiple authors II

How to build & run your first livestream with 100ms Live-streaming SDK

Video conferencing has been around forever. But what used to be the sporadic coast-to-coast team catch-up, or the predominantly inside sales call became a household necessity with the 2020 Covid-19 Pandemic. Everyday work. Family reunions. Even doctor consults, workout trainers, and online astrologers. Oh, and never forget the kids jumping in the background, drowning in the virtual class they were supposed to attend.

That said, smart businesses have been quick to offer video-enabled communication services since, and the trend just seems to be getting started, whether it’s telehealth, test-prep, dating, or shopping. While businesses can obviously look at building this audio-video infra in-house, most find it time and resource-intensive.

Of course, building a scalable video infrastructure from scratch is no mean feat, unless that’s the primary focus you want for your engineering team. Luckily there are at least a handful of Video SDK providers that offer best-in-class video infrastructure.

But then, how do you decide which one works best for you?  

**To answer that, we decided to put together this Buying Guide.** 

First, we’ve handpicked the best of the best video SDKs based on customer reviews, product usage, and capabilities offered. And then, we battled them out on **features & functionality, compliance & security, support, and pricing**. 

From features, time to respond, implementation help, and total cost of ownership, you should find all the details you need to make an informed buying decision right here. 

The Buying Guide comprises four separate articles. Each article focuses on comparing vendors across a single, relevant parameter. All information for said comparison has been obtained from each vendor’s publicly available documentation.

---

Here is a quick summary of each article. Click on the article link for a deep dive. 

## **Functionality and Features**

Each audio-video SDK offers specific features and functions that enable its customers to meet their goals. We’ve listed out the basic offerings across **Agora, Twilio, Jitsi (Jitsi as a Service), Zoom, and 100ms** below. 

However, you can go deeper and explore how vendors provide specific features such as **streaming out with RTMP and HLS, active speaker detection, chat, polls, whiteboard, hand raise, and more** in our detailed article.

**Full article**: [Features & Functionality for for Agora, Twilio, Jitsi (JaaS), Zoom & 100ms](https://www.100ms.live/blog/buying-guide-features)

### Agora

Users can build the video-calling feature using the SDK, while interactive live streaming can be built using Agora’s Live Streaming SDK. Call recording can be enabled using the dashboard and API. 

Additionally, a noise reduction feature can be built using an additional integration. Agora also has a virtual background extension that allows for background modification. Background blur can be implemented using the SDK.

### Twilio

With Twilio, users can build the video-calling feature using the SDK, while interactive live streaming can be built with Twilio Live. As for noise reduction, it can be built using the SDK. Call recording can be enabled using the dashboard and API.  Background modifications can be built using the Twilio video processor SDK.

### Jitsi (Jitsi as a Service)

Jitsi users can build video-calling, noise reduction, call recording, and background modification using the SDK. With regard to interactive live streaming, there is no explicit mention of it in the JaaS documentation. 

### Zoom

With Zoom, we have examined the feature offerings of both the Zoom Video SDK as well as the Meeting SDK. 

**Zoom Video SDK**
Users can build video-calling, noise reduction, and background modification features using the SDK. Call recording can be enabled using the dashboard and API. There is no explicit mention of interactive live streams in the Zoom documentation.

**Zoom Meeting SDK**
Users can build video-calling, noise reduction, and background modification features using the SDK. Call recording can be enabled using the dashboard and API. For interactive live streaming, Zoom allows for the streaming of sessions with up to 10k participants.

### 100ms

100ms enables users to build video-calling, interactive live-streaming (HLS + WebRTC in a single SDK), background modification and much more using the SDK. Noise reduction is currently available in Beta. 100ms also allows for instant streaming of video conferencing sessions with up to 10k participants.

---

## **Compliance and Security**

With data thefts and security breaches becoming more common with each passing day, it’s important for decision-makers to understand the security features each audio-video SDK offers. To help with that, we’ve listed the compliance certifications that each audio-video infra provider holds, below.

However, for more details about security features such as **access control, enterprise authentication, end-to-end encryption, privacy and encryption of recordings, and audit trails,** please take a look at our detailed article.

**Full article:** [Compliance & Security for Agora, Twilio, Jitsi (JaaS), Zoom & 100ms](https://www.100ms.live/blog/buying-guide-compliance-and-security)

### Agora

1. GDPR
2. HIPAA
3. CCPA
4. COPPA
5. ISO/IEC 27001
6. SOC 2

### Twilio

1. GDPR
2. ISO 27001
3. AICPA SOC 2 
4. HIPAA

### Jitsi

1. HIPAA
2. GDPR compliant for data processors

### Zoom

1. SOC 2 Type II
2. GDPR
3. HIPAA
4. ISO/IEC 27001:2013

### 100ms

1. SOC2 Type 1 & SOC 2 Type 2

---

## **Support**

Vendor support is as important a factor as the features they provide. Integration and post-integration support extended by these providers will help you when something breaks on the backend, and you need expert support. 

While you can find a short summary explaining the support extended by Agora, Twilio, Jitsi, Zoom, and 100ms below, we’ve also created an elaborate comparison to give you a full overview in terms of **Cost of Support**, **Integration/Account Management Support, Post-Integration Support,** and **Community Support.**

**Full article**: [Support for Agora, Twilio, Jitsi (JaaS), Zoom & 100ms](https://www.100ms.live/blog/buying-guide-support)

### Agora

Agora has different paid support plans - Standard, Premium, and Enterprise. Apart from these, the platform offers one free support plan.

### Twilio

Twilio has four different support plans — Developer (free), Production (paid), Business (paid), and Personalized (paid). As for the plans, support is based on the volume of usage.

### Jitsi as a Service (JaaS)

Jitsi’s open-source version is supported by a large community - the Jitsi community forum. Apart from this, Jitsi’s paid version, 8x8 Jitsi as a Service, includes dedicated support for strategic customers, as explained in their Global Premium Plus Support plan. 

### Zoom

Zoom offers dedicated support for developers via Premier Developer Support which offers prioritized developer-specific resources. This guide does not explore Zoom support plans aimed at non-dev users and admins.

### 100ms

All support functions are available to paying customers at no additional cost. 100ms also offers testing support at no extra cost. This includes user testing, load testing, and network/device stress-testing.

---

## **Pricing**

Pricing is one of the most crucial deciding factors when it comes to purchasing an SDK. As with the other parameters, here’s a quick overview of what each vendor charges.

However, details on each vendor’s pricing policies are beyond the scope of this piece. We’ve put together a detailed guide that explains the **pricing models, pricing for recording, live streaming, add-ons,** and more for each of these providers.

**Full Article:** [Pricing for Agora, Twilio, Jitsi (JaaS), Zoom & 100ms](https://www.100ms.live/blog/buying-guide-pricing)

### Agora

Agora’s pricing is based on usage, which includes the number of minutes used, the number of users, and the unit price. However, Agora follows pricing on the basis of aggregate resolution in calls - this has been explained in detail in the main article. 

The unit price per 1,000 minutes is $0.99 (for audio) and $3.99 (for video HD). Pricing for Video Full HD, Video 2K, and Video 2K+ is explained in detail within the main article.

### Twilio

Pricing for Twilio scales with participant minutes. We have examined the pricing of two Twilio video products - Twilio P2P & Twilio Video Groups.

**Twilio P2P**
Allows up to 3 participants and up to 10 audio-only participants. Priced at $0.0015 per participant per minute. 

**Twilio Video Groups**
Allows users to create video apps for up to 50 participants. Priced at $0.004 per participant minute - defined only by the minutes a user spends connected in a room.

### Jitsi (Jitsi as a Service)

Jitsi’s pricing is based on a per-user model and charges for monthly active users (MAU). According to Jitsi, an MAU is a user who attends at least one meeting with at least one other user within a particular month. An MAU is also tracked on the basis of the device they log in from.

JaaS offers various plans, and the pricing for each plan varies depending on the MAU. The JaaS Dev plan allows up to 25 MAU free, and only add-ons are charged extra. Under the JaaS Basic plan, pricing is $99 per month for 300 MAUs. The pricing for JaaS Standard, JaaS Business, and plans with more than 3000 MAU are mentioned in the main article.

### Zoom

Zoom offers two SDKs: a Video SDK(charged on the basis of usage) and a Meeting SDK (charged on per user basis). 

**Zoom Meeting SDK**
The Meeting SDK offers four paid tiers: Basic, Pro, Business, and Enterprise. To use the Meeting SDK, the host is the only person who must purchase and hold a license. This license carries a specific limit on the number of participants who can be supported in each tier. 

**Zoom Video SDK**
Under Zoom Video SDK, there are two pricing levels. With pay-as-you-go, you get 10,000 minutes per month, after which the plan is priced at $0.0035 per minute. At the second level, you pay $1000 per year with 30,000 minutes included per month. After the limit is crossed, you pay $0.003 per minute.

### 100ms

100ms offers a single SDK offering both video conferencing and live streaming capabilities with straightforward pricing for both use cases. 

100ms offers 10,000 free minutes for conferencing and another 10,000 free minutes for streaming for each business every month in addition to 1000 free encoding minutes. 

Beyond this, video conferencing is charged at $0.004 per participant per minute, while audio-only calls are charged at $0.001 per participant per minute. Live Streaming costs $0.004 per broadcaster and $0.0012 per viewer per minute, while additional encoding minutes are charged at $0.04. 

---

While there are various parameters and aspects that PMs, CTOs, and decision-makers need to keep in mind, these are a few that we considered indispensable. We recommend that you have a look at the separate pieces focusing on each of these parameters. They offer research-based data from each vendor, as well as helpful links you can use to conduct in-depth research yourself. 

To know more about how 100ms can help fill in your video conferencing requirements, [book a call with us](https://meet.100ms.live/meetings/isha-deo/intro).

thumbnail_Blog - 6.png

Blog - 6.png

Buying Guide: Agora, Twilio, Jitsi (JaaS), Zoom & 100ms

We know that WebRTC powers almost every kind of live video solution imaginable these days. And not using a TURN server in a WebRTC-based solution, can be compared to not bringing a spare car tyre to a trip to the Himalayas. It may not look like a big deal at first, but in reality, you might actually need more than one spare tyre to complete the trip.

It is estimated that almost 20% of WebRTC call connections require a TURN server, whatever may the [architecture of the application](https://www.100ms.live/blog/webrtc-server-side) be. Even the latest WebRTC ingest and egress standards—[WHIP](https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html) and [WHEP](https://www.ietf.org/id/draft-murillo-whep-00.html) make use of STUN/TURN servers. In this article, we’ll discuss everything you need to know about STUN and TURN servers. We will also try to deploy and test a popular open-source TURN server in the end.

## The need for STUN/TURN Servers

Depending on the architecture, **WebRTC** creates a connection between peers directly or with a media server in the middle. But sometimes, a direct connection to other peers or the media server itself may fail at times. This can happen mainly because of 2 reasons—**NAT** and **Firewall**. We will briefly look into them here.

### NAT (Network Address Translation)

The **Network Address Translation** is a mechanism that maps the public transport address into private transport address of a device within its local network and vice-versa. A **NAT** usually lives on a router that sits between the local network and the internet.

> A **Transport address** is nothing but the combination of an IP address and a port number. It is used to uniquely identify a device for communication.
> 

**NAT** was designed to offer security and help conserve IPV4 addresses over the internet. But it also came with a few issues in respect to direct connection:

1. The host device behind a NAT only knows its own private IP address. This makes it difficult to communicate with others outside the device’s local network, since the private IP address (and transport address) is basically useless elsewhere.
2. **Hole punching** is not supported when the target NAT’s mapping behaviour is “address-dependant” or “address-and-port-dependant”, as defined in [RFC4787](https://datatracker.ietf.org/doc/html/rfc4787#section-4.1) standard. In short, hole punching is a networking technique generally used for establishing a direct peer-to-peer (P2P) connection.

### Firewall

Every host device has its own firewall that regulates all the incoming and outgoing network traffic for security purposes. Many enterprise firewalls simply block **UDP** traffic, as it is not as secure as other transport layer protocols such as **TCP**. 

Since, **WebRTC** uses **UDP** for all of its media sessions to optimize for performance, there is a high chance that it may be blocked too. Sometimes, the firewall might also find other peers suspicious, blocking them from establishing a direct connection. 

**WebRTC** uses **ICE (Interactive Connection Establishment) framework** to find the best path to connect the peers. When the direct connection fails, there must be a way for the peers to establish a connection.  This is where **[STUN](https://www.rfc-editor.org/rfc/rfc8489.html)** and **[TURN](https://www.rfc-editor.org/rfc/rfc8656)** come into picture. Based on several parameters, the ICE framework will automatically try to use the configured **STUN** and/or **TURN** servers, when a direct connection is impossible. Since, how ICE works is out of the scope of this article, let us focus only on the **STUN** and **TURN** servers.

## Intro to STUN and TURN Servers

To understand how these servers help in establishing a connection, let us first try to understand what a STUN server and a TURN server does. 

### STUN Server

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/stun_server_working_a63e0a2694/stun_server_working_a63e0a2694.gif) 

**STUN** is a network protocol used to retrieve the public IP address (or Transport address) of a device behind NAT so that the device can communicate after knowing its address on the internet. Usually, a **STUN** server is requested by the peer to know its public IP addresses (or specifically its transport address). Then, the retrieved address is used to communicate with devices outside its local network.

### TURN Server

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/turn_server_working_46b526d316/turn_server_working_46b526d316.gif) 

Sometimes, a direct connection is not possible due to NAT restrictions or a firewall blocking a connection, even with the help of STUN server. The only way for peers is to communicate in this case, is by routing the media through a relay server. A TURN server is used for cases like this.

**TURN** is a network protocol that enables a server to relay data packets (media in this case) between peers when no direct connections are possible. The peers may route media to the TURN server which will relay it to the other peer continuously.

> **TURN** servers are usually referred to as **Relay Servers**. According to the [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) specification, TURN is considered an extension to the STUN. So, a typical TURN server would include a STUN server implementation by default.
> 

## Internal Working of a TURN Server

Before trying to understand the operations of a TURN server, let’s take a look at some terminologies involved.

There are 2 participants associated with a TURN session:

1. **Client (TURN Client)** - A host attached to the TURN server for communication. It interacts with the server following the specification by sending TURN messages.
2. **Peers** - The hosts that the client wishes to communicate with. The TURN server relays the traffic between the client and the peers. Every peer interacts with the server like it usually does with other peers.

The Transport Addresses associated with TURN client and Peers:

1. **Host Transport Address** - The actual transport address of a client or a peer. It consists of the private IP address within its local network.
2. **Server-Reflexive Transport Address** - The transport address of a client or a peer that is visible from outside of its NAT. It consists of the public IP address outside the local network. This address is allocated by the NAT to correspond to a specific host transport address.

The Transport Addresses associated with a TURN server:

1. **Server Transport Address** - The transport address of the TURN server that is used for sending TURN messages to the server. This is the transport address that the client uses to communicate with the server.
2. **Relayed Transport Address** - The transport address of the TURN server that is used for relaying packets between the client and a peer. A peer sends data to this address on the TURN server, and the packet is then relayed to the client.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/turn_server_usage_cfea324fa0/turn_server_usage_cfea324fa0.jpg) 

### Summary of TURN server Operations

The TURN client already has access to the Server Transport Address of the TURN server, from ICE server configuration in case of WebRTC. The TURN server can only sense the Server-Reflexive Transport Address of the client (and peer) and not the Host Transport Address. 

This happens when there’s a NAT in the middle, that maps the Server-Reflexive Transport Address to the Host Transport Address for the client/peer. When there’s no NAT in the connection, there’s no need to differentiate the addresses and it can simply be called Transport Address.

Every TURN client is assigned its own Relayed Transport Address for relaying the data. The peers will receive the data from this address and any data that the peers send to this address will be relayed to the respective TURN client that was allocated that address.

### Transport Protocols

The following table illustrates the protocols used for transport between a TURN client to a server and TURN server to a peer.

| TURN client to TURN server | TURN server to peer |
| --- | --- |
| UDP | UDP |
| TCP | UDP |
| TLS-over-TCP | UDP |
| DTLS-over-UDP | UDP |

According to [RFC8656](https://www.rfc-editor.org/rfc/rfc8656#name-transports) specification, the communication between the server and a peer always uses UDP. This is very helpful in case of WebRTC. Since the media sessions use UDP for transport, a peer doesn’t have to differentiate between another peer and the TURN server to communicate with it.

And if the client blocks UDP traffic completely, there’s always room to use other transport protocols like TCP, TLS-over-TCP and DTLS-over-UDP. But, what happens if both the client and a peer happen to block UDP traffic? We will come back to that in a later section.

Now let us try to understand the exact steps involved in using a TURN server to achieve connectivity among peers. The peer that cannot participate in a direct connection is usually the TURN client and it will initiate the process.

### 1. Preparing to Send Data to Peer

A client can use TURN commands to create and manipulate an ALLOCATION on the TURN server, by communicating with the server on its Server Transport Address. An ALLOCATION is basically a “TURN Session” associated with the client. Among other things, every ALLOCATION has a corresponding Relayed Transport Address to be used.

The Relayed Transport Address is the address that the other peers can use to interact with. A peer can use the address to relay the data to client. Additionally, the source of the data from the client, relayed to the peer will be the Relayed Transport Address from the peer’s perspective.

### 2. Preparing to Receive Data from Peer

When the peers send data to the Relayed Transport Address, it’ll be matched with the ALLOCATION and be relayed to the client who is associated with it. But the data is discarded until a PERMISSION is created for the peer to send data to the client.

The PERMISSION for a peer can be created by the client, by using the address of the peer as it appears to the TURN server. The peer can get this address by sending a STUN Binding Request to the same TURN server (not any other STUN/TURN server) and then forward send it to the client by other means. ICE Candidates are used to handle this in case of WebRTC. Once this is done, the peer can start communicating with the client through the TURN server.

> The reason that the peer has to send a STUN Binding Request to the same server is, the transport address the peer got from a different STUN server, may not work for the target TURN server. This happens when the mapping behaviour is either “address-dependant” or “address-and-port-dependant”.
> 

### 3. Sending and Receiving Data from Peers

Now, the TURN is ready for relaying data between the client and peers. There are 2 mechanisms for sending and receiving data from the peers through the relay server.

1. **Send and Data Mechanism**

In this mechanism, **SendIndication** and **DataIndication** messages are used to relay data between the client and the peers.
The client sends a **SendIndication** message to the server which contains the data and the peer information the data has to be relayed to. The server forms a UDP Datagram using that message and sends it to the respective peer.

Similarly, when a peer sends a UDP Datagram to the server, it is then converted into a **DataIndication** message and is relayed to the client.

 b.  **Channel Mechanism**

In this mechanism, a channel is created for communication between a client-peer pair by using both their Transport Addresses once. There is no need to specify the Transport Address when sending data, unlike in the Send and Data mechanism. This mechanism is recommended when there’s a need for sending lots of data, mainly due of its smaller packet size.

The client creates a channel by sending a **ChannelBind** request with the peer’s Transport Address and a unique **ChannelId**. The communication between the client and the server is done by use of **ChannelData** messages that contain a **ChannelId** and data among other things. The **ChannelData** message is converted to a UDP Datagram and sent to the peer. Similarly, any data received from the peer corresponding to a channel is packaged into a **ChannelData** message before being relaying to the respective client.

### 4. Maintaining the TURN Session

Every ALLOCATION has a LIFETIME attribute, denoting the time after which it will expire. The client has to send a refresh request to keep its ALLOCATION alive.

## TURN Usage

There are mainly 2 ways a TURN server can be used to achieve connectivity. We will look into them by considering one-on-one connection as an example.

1. One client with a single Allocation and one peer.
2. Two clients with respective Allocations.

### One client with a single Allocation and one peer

In this case, there is one TURN client and one peer. Usually, when only one party has issues with the direct connection, this is the recommended way.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/turn_server_single_allocation_f02b45935a/turn_server_single_allocation_f02b45935a.jpg) 

### Two clients with respective Allocations

In this case, both parties are TURN clients communicating with each other. This is the solution when both the parties don’t support direct connection and that includes blocking UDP traffic.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/turn_server_two_allocation_47008ca4e7/turn_server_two_allocation_47008ca4e7.jpg) 

Now that we’ve seen too much theoretically about TURN server, let us also try to setup and test our own TURN server. We will be using coTURN, a really popular open-source TURN server implementation.

## Setting up a TURN Server with coTURN

In this section, we will set up our own TURN server on Ubuntu (and similar Linux based OSes), using coTURN. We will do it in the following steps:

1. Installing coTURN
2. Configuring coTURN
3. Testing the TURN server

### 1. Installing coTURN

First, create an AWS EC2 instance or a GCP Cloud Compute instance of Ubuntu OS. We will be installing hosting the TURN server in this instance. Then, connect to the instance with the help of SSH.

Run the following commands to update the repo and install coTURN.

`sudo apt-get update`

`sudo apt-get install coturn`

To make sure that coTURN always runs at startup, edit the file */etc/default/coturn.*

`sudo nano /etc/default/coturn`

Inside the file, find the line `TURNSERVER_ENABLED=1` and uncomment it. Now start the coTURN server by running the following command.

`systemctl start coturn`

### 2. Configuring coTURN

Before modifying the configuration file it is recommended to copy the original configuration file in case something goes wrong.

`mv /etc/turnserver.conf /etc/turnserver.conf.backup`

`sudo nano /etc/turnserver.conf`

Now, proceed to edit the configuration file and replace it with the below settings.

```
realm=<Your Domain>
server-name=turnserver
fingerprint

listening-ip=0.0.0.0
external-ip=<External IP Address>
listening-port=3478
min-port=10000
max-port=20000

user=testname:testpass
lt-cred-mech

log-file=/var/log/turnserver.log
verbose
```

After finishing the changes, you can restart the coTURN server with the following command.

`sudo service coturn restart`

### 3. Testing the TURN server

Now that we’ve installed and configured a TURN server with coTURN, it is time to test if it works correctly. One of the easiest ways to do this is to use [this site that checks for Trickle ICE functionality](https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/).

First add the STUN and TURN information in the in the following format. For STUN server URI `stun:<IP Address>:<Port>` and for TURN URI `turn:<IP Address>:<Port>` with the username and password configured before. Then, click on “Add Server” button.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/testing_turn_server_709db3bd07/testing_turn_server_709db3bd07.png) 

Once everything is set, you can start the test by clicking on the “Gather Candidates” button.

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/testing_turn_server_result_d34111cc2d/testing_turn_server_result_d34111cc2d.png) 

If your results look similar to the one in the image above without any fatal errors below the button, then the TURN server is working as expected.

> Special Mention: If you’re trying to build a custom TURN server/client tuned for your use case, check out [pion TURN](https://github.com/pion/turn). It is open source and has great support from the community.
> 

Now that you know everything you need about TURN servers and the role it plays in your WebRTC call, you might as well try building a [simple WebRTC app with this guide I wrote](https://www.100ms.live/blog/webrtc-python-react-app).

## Reference

- [Connecting | WebRTC for the Curious](https://webrtcforthecurious.com/docs/03-connecting/#turn)
- [RFC 5766 - Traversal Using Relays around NAT (TURN): Relay Extensions to Session Traversal Utilities for NAT (STUN) (ietf.org)](https://datatracker.ietf.org/doc/html/rfc5766)
- [RFC 8656 - Traversal Using Relays around NAT (TURN): Relay Extensions to Session Traversal Utilities for NAT (STUN) (ietf.org)](https://datatracker.ietf.org/doc/html/rfc8656)

large_turn-server-usage.jpg

medium_turn-server-usage.jpg

small_turn-server-usage.jpg

thumbnail_turn-server-usage.jpg

turn-server-usage.jpg

WebRTC TURN Server: Internals and Beyond

Trying to migrate your tech stack from Agora to 100ms?

This guide will help you do exactly that. But first, let’s look at the differences between Agora and 100ms’ Web SDKs — **Agora Video JS SDK** and **100ms JS SDK**, from a technical perspective. We’ve also offered a comparison between them at the code-level to help you scope code changes for your migration.

|  | **Agora** | **100ms** |
| --- | --- | --- |
| **Rooms and Authentication** | Agora uses the concept of channels. `joinChannel` call is made to create and join a channel. App clients that pass the same channel name join the same channel. <br> A token is a credential that authenticates a user when an app client joins a channel. | 100ms uses the concept of rooms. The rooms are identified by a “Room Id” that is generated automatically. <br> Clients are authenticated to join the room requested with the `room id` and configuration, using the `auth token` |
| **Peers** | Every user in a channel has a unique user ID. To access the local device methods `LocalTrack` object is used. For every other participant in the room, `RemoteTrack` object is used. | Every peer corresponds to a `Peer` object including the local device. <br>The methods related to the local device are associated with `HMSActions` that directly connect to the 100ms backend. |
| **Media** | Every user can create a set of `Tracks` and publish them to the channel. Objects of type `LocalTrack` and `RemoteTrack` are used for sending/managing audio and video tracks. UI can be rendered by listening to every individual track with callback functions. | No complex concept like “tracks” for handling media. Straightforward methods available on `HMSActions` to enable/disable camera, microphone, screen-share.<br>Easy to render UI with appropriate `Selectors` and callback functions. |
| **Permissions and Pub/Sub** | The type of user role determines whether the user in the channel has streaming permissions. A user role is either host or audience. | `Roles` can be used to define permissions for a peer, including Pub/Sub permissions, permission to mute/unmute another peer in the same room & more. |
| **Room Configuration** | The channels have to be individually configured every time they are created. | A `Template` consists of all configurations for a room, along with the possible `Roles` to define peer behaviour. <br>When a room is created dynamically, it can be configured with a pre-defined or a custom `Template` by referring to its ID or name. Easy to create your own [Templates from the Dashboard](https://dashboard.100ms.live/templates). |

Now, we can dive into the process of migrating your app to 100ms.

## Migrating to 100ms SDK

Once you’ve explored the basic concepts of the 100ms SDK and how it compares with the Agora Video SDK, you’re all set to start the migration process. The first step is to create a project.

### Creating your Project

To help you get started by creating your first 100ms project, we have a product walkthrough by Arjun, where he demonstrates how to create a project from the Dashboard.

<iframe width="100%" height="380" src="https://www.youtube.com/embed/bq9NQLTX4uM" title="100ms Product Walkthrough" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Once an app is created on the dashboard, we have to refactor the code in order to migrate the app from the Agora Video SDK to the 100ms SDK. 

Here, refactoring involves changing just a few lines of code.

### 1. Joining the Room

```jsx
<form id="join">
    <h2>Join Room</h2>

    // Channel name (required for Agora)
    <div class="input-container">
        <input id="cname" type="text" name="channelname" placeholder="Channel name" />
    </div>

		// Username (required for 100ms)
    <div class="input-container">
        <input id="uname" type="text" name="username" placeholder="Username" />
    </div>

		// Auth token (required for both Agora and 100ms)
    <div class="input-container">
        <input id="token" type="text" name="token" placeholder="Auth token" />
    </div>
    <button type="button" class="btn-primary" id="join-btn">Join</button>
</form>
```

The **100ms** SDK gives you access to abstracted objects like `hmsStore` and `hmsActions`  so as to easily manage the state of your room. We’ll see how these objects are used in the upcoming sections.

**Agora**

```jsx
const client = AgoraRTC.createClient({
  codec: "vp8",
  mode: "rtc",
});

// Join room on button click
document.getElementById("join-btn").onclick = () => {
		client.join({
				APPID: "Specify an APPID for your Project", 
				CHANNEL: document.getElementById("cname").value, 
				TOKEN: document.getElementById("token").value,
		});
};
```

**100ms**

```jsx
// Initialize HMS Store
const hmsManager = new HMSReactiveStore();
hmsManager.triggerOnSubscribe();
const hmsStore = hmsManager.getStore();
const hmsActions = hmsManager.getHMSActions();

// Join room on button click
document.getElementById("join-btn").onclick = () => {
    hmsActions.join({
        userName: document.getElementById("uname").value,
        authToken: document.getElementById("token").value,
				...otherConfig
    });
};
```

### Authentication
We can use the dashboard to create a temporary token that will automatically expire in 24hrs. You can follow [this guide to generate tokens on your server](https://www.100ms.live/docs/server-side/v2/foundation/authentication-and-tokens).

> Auth tokens are generally issued by the server upon request from the client application.
> 
1. In your **Dashboard**, go to the **Rooms** section. Now, click on the **RoomId** of the room you want to generate a **token** for.

 ![](https://storage.googleapis.com/100ms-cms-prod/cms/100ms_dashboard_rooms_3f363b31c8/100ms_dashboard_rooms_3f363b31c8.png) 

2. Click on the **Join room** button.

 ![](https://storage.googleapis.com/100ms-cms-prod/cms/100ms_room_details_8a16a98edb/100ms_room_details_8a16a98edb.png) 

3. Now, by clicking the **key** icon, you can generate and copy the **token** for your room based on the **role**.

 ![](https://storage.googleapis.com/100ms-cms-prod/cms/100ms_room_token_generate_eefe418af7/100ms_room_token_generate_eefe418af7.png) 

4. Once this **token** has expired (after 24hrs), you can generate another token by following the same method.

### 2. Rendering Audio/Video

In **100ms**, the concept of “tracks” is abstracted to provide the best developer experience and avoid mistakes that pop up when handling them explicitly. The `hmsStore.subscribe` is automatically called on every peer join/leave event. This means that you don’t have to worry about every single edge-case for a peer joining and leaving the room. 

**Agora**
```
    // Listen for the "user-published" event, from which you can get an AgoraRTCRemoteUser object.
    rtc.client.on("user-published", async (user, mediaType) => {
        // Subscribe to the remote user when the SDK triggers the "user-published" event
        await rtc.client.subscribe(user, mediaType);
        console.log("subscribe success");

        // If the remote user publishes a video track.
        if (mediaType === "video") {
            // Get the RemoteVideoTrack object in the AgoraRTCRemoteUser object.
            const remoteVideoTrack = user.videoTrack;
            // Dynamically create a container in the form of a DIV element for playing the remote video track.
            const remotePlayerContainer = document.createElement("div");
            // Specify the ID of the DIV container. You can use the uid of the remote user.
            remotePlayerContainer.id = user.uid.toString();
            remotePlayerContainer.textContent = "Remote user " + user.uid.toString();
            remotePlayerContainer.style.width = "640px";
            remotePlayerContainer.style.height = "480px";
            document.body.append(remotePlayerContainer);

            // Play the remote video track.
            // Pass the DIV container and the SDK dynamically creates a player in the container for playing the remote video track.
            remoteVideoTrack.play(remotePlayerContainer);
        }

        // If the remote user publishes an audio track.
        if (mediaType === "audio") {
            // Get the RemoteAudioTrack object in the AgoraRTCRemoteUser object.
            const remoteAudioTrack = user.audioTrack;
            // Play the remote audio track. No need to pass any DOM element.
            remoteAudioTrack.play();
        }

        // Listen for the "user-unpublished" event
        rtc.client.on("user-unpublished", user => {
            // Get the dynamically created DIV container.
            const remotePlayerContainer = document.getElementById(user.uid);
            // Destroy the container.
            remotePlayerContainer.remove();
        });
    });

    window.onload = function () {
        document.getElementById("join").onclick = async function () {
            // Join an RTC channel.
            const uid = await rtc.client.join(options.appId, document.getElementById("cname").value, document.getElementById("token").value);
            // Create a local audio track from the audio sampled by a microphone.
            rtc.localAudioTrack = await AgoraRTC.createMicrophoneAudioTrack();
            // Create a local video track from the video captured by a camera.
            rtc.localVideoTrack = await AgoraRTC.createCameraVideoTrack();
            // Publish the local audio and video tracks to the RTC channel.
            await rtc.client.publish([rtc.localAudioTrack, rtc.localVideoTrack]);
            // Dynamically create a container in the form of a DIV element for playing the local video track.
            const localPlayerContainer = document.createElement("div");
            // Specify the ID of the DIV container. You can use the uid of the local user.
            localPlayerContainer.id = uid;
            localPlayerContainer.textContent = "Local user " + uid;
            localPlayerContainer.style.width = "640px";
            localPlayerContainer.style.height = "480px";
            document.body.append(localPlayerContainer);

            // Play the local video track.
            // Pass the DIV container and the SDK dynamically creates a player in the container for playing the local video track.
            rtc.localVideoTrack.play(localPlayerContainer);
            console.log("publish success!");
        };
    };
```
**100ms**

```
// Callback for handling peer rendering - called on peer join/leave events
hmsStore.subscribe(renderPeers, selectPeers);

// Display a tile for each peer in the peer list
function renderPeers() {
    document.getElementById("peers-container").innerHTML = "";
    const peers = hmsStore.getState(selectPeers);

    peers.forEach((peer) => {
        if (peer.videoTrack) {
            document.getElementById("peers-container").append(renderPeer(peer));
        }
    });
}

// Render a single peer
function renderPeer(peer) {
		const peertile = document.createElement('peer-tile');
    peertile.id = peer.id;
    peertile.innerText = peer.name;

    const videoElement = document.createElement('peer-video');
		videoElement.className = 'video';
    videoElement.autoplay = true;
    videoElement.muted = true;
    videoElement.playsinline = true;

    hmsActions.attachVideo(peer.videoTrack, videoElement);

    peerTileDiv.append(videoElement);
    return peerTile;
}
```

### 3. Mute and Unmute Audio/Video

```
<div id="controls" class="control-bar">
    <button id="mute-aud" class="btn-control">Mute</button>
    <button id="mute-vid" class="btn-control">Hide</button>
		...
</div>
```

100ms offers simple yet consistent methods for performing common operations like muting/unmuting audio and switching video on/off.

**Agora**
```
// Global mute states
let audioEnabled = true;
let videoEnabled = true;

// Mute and unmute audio
document.getElementById("mute-aud").onclick = () => {
		audioEnabled = !audioEnabled;
	  localTracks.audioTrack.setEnabled(audioEnabled);	
    document.getElementById("mute-aud").textContent = audioEnabled ? "Mute" : "Unmute";
};

// Mute and unmute video
document.getElementById("mute-vid").onclick = () => {
		videoEnabled = !videoEnabled;
		localTracks.videoTrack.setEnabled(videoEnabled);
    document.getElementById("mute-vid").textContent = videoEnabled ? "Hide" : "Unhide";
};
```

**100ms**
```
// Mute and unmute audio
document.getElementById("mute-aud").onclick = () => {
    const audioEnabled = !hmsStore.getState(selectIsLocalAudioEnabled);
    hmsActions.setLocalAudioEnabled(audioEnabled);
    document.getElementById("mute-aud").textContent = audioEnabled ? "Mute" : "Unmute";
};

// Mute and unmute video
document.getElementById("mute-vid").onclick = () => {
    const videoEnabled = !hmsStore.getState(selectIsLocalVideoEnabled);
    hmsActions.setLocalVideoEnabled(videoEnabled);
    document.getElementById("mute-vid").textContent = videoEnabled ? "Hide" : "Unhide";
};
```

### 4. Leaving the room

```
<div id="controls" class="control-bar">
		...
    <button id="leave-btn" class="btn-control">Leave</button>
</div>
```

**Agora**

```
document.getElementById("leave-btn").onclick = async function () {
            // Destroy the local audio and video tracks.
            rtc.localAudioTrack.close();
            rtc.localVideoTrack.close();

            // Traverse all remote users.
            rtc.client.remoteUsers.forEach(user => {
                // Destroy the dynamically created DIV containers.
                const playerContainer = document.getElementById(user.uid);
                playerContainer && playerContainer.remove();
            });

            // Leave the channel.
            await rtc.client.leave();
        };
```

**100ms**

```
document.getElementById("leave-btn").onclick = function () {
    hmsActions.leave().then(() => console.log("User left the room"));
}
```
### 5. User roles and managing permissions

In Agora, the type of user role determines whether the user in the channel has streaming permissions. It is either host or audience.

- host - The host refers to a user who has streaming permissions in the channel.
- audience - The audience are users who do not have streaming permissions in the channel. The audience can only subscribe to the remote audio and video streams, but cannot publish the audio and video streams.

In 100ms, it is easier to manage user permissions and create custom roles to fit your use case better. Understand roles and how they can help better [here](https://www.100ms.live/blog/roles-on-100ms).

- When creating a project, you can choose to create a ‘Custom App’.
- Click on ‘Create a custom role’ to get started defining custom roles for your app.
 ![](https://storage.googleapis.com/100ms-cms-prod/cms/Screenshot_2022_09_28_at_4_05_52_PM_9feede3268/Screenshot_2022_09_28_at_4_05_52_PM_9feede3268.png) 
- Edit the name of the new role and modify the ‘Publish Strategy’. Whether the peer with the role should be able to share audio/video/screen or not.
 ![](https://storage.googleapis.com/100ms-cms-prod/cms/Screenshot_2022_09_28_at_4_05_57_PM_594f3d2191/Screenshot_2022_09_28_at_4_05_57_PM_594f3d2191.png) 
- Define the permissions for the role. Should the users with the new role be allowed to mute participants, remove them from meetings, start/end sessions, etc.
 ![](https://storage.googleapis.com/100ms-cms-prod/cms/Screenshot_2022_09_28_at_4_06_07_PM_a080a2cb8b/Screenshot_2022_09_28_at_4_06_07_PM_a080a2cb8b.png) 

## Demo

You can explore our [Javascript Quickstart](https://www.100ms.live/docs/javascript/v2/guides/javascript-quickstart) example below, implemented with a similar source code.

<iframe src="https://codesandbox.io/embed/hms-js-quickstart-forked-0ffu31?fontsize=14&hidenavigation=1&theme=dark"
     style="width:100%; height:500px; border:0; border-radius: 4px; overflow:hidden;"
     title="hms-js-quickstart (forked)"
     allow="accelerometer; ambient-light-sensor; camera; encrypted-media; geolocation; gyroscope; hid; microphone; midi; payment; usb; vr; xr-spatial-tracking"
     sandbox="allow-forms allow-modals allow-popups allow-presentation allow-same-origin allow-scripts"
   ></iframe>

Migrating to 100ms SDK is a simple task and, as displayed by the comparison above, well worth the effort in the long run.

## Next Steps

If you’re interested in building production-grade apps from scratch with 100ms, here are some resources you might be interested in:

- [Javascript Quickstart](https://www.100ms.live/docs/javascript/v2/guides/javascript-quickstart)
- [Server-side Implementation (Authentication & Webhooks)](https://www.100ms.live/docs/server-side/v2/foundation/basics)

Try the 100ms SDK for yourself, and you’ll know exactly how it can empower your video app to perform better with less developer effort. 



large_Blog Cover.png

medium_Blog Cover.png

small_Blog Cover.png

thumbnail_Blog Cover.png

Blog Cover.png

Comparing the Code: Migrating from Agora to 100ms [Web Edition]

Are you on the hunt for a live video solution to replace your current tech stack powered by **Agora** or **Twilio**? **100ms** might just be the alternative you are looking for. With truly scalable, reliable and dev-friendly live video SDKs, 100ms is your new best bet for **Real-time Video Conferencing** and **Interactive Live Streaming**.

The **100ms** SDK comes with everything live video stuffed into a single lightweight package. You don’t have to worry about switching SDKs for implementing specific features like live streaming, chat or virtual background in your app. In this article, we will be comparing **Twilio**, **Agora** and **100ms** in terms of the feature set you can build around with, and talk about what 100ms does differently that you might end up loving it.

## Why consider 100ms?

To start with, here are a few reasons to consider **100ms** over **Twilio’s** and **Agora’s** video SDKs:

- **200X Participants in a single room:** With **100ms**, you can add up to **10,000** participants in a room with **100** participants publishing their audio/video at the same time.
- **Transition from Conferencing to Live Streaming:** 100ms lets you transition from an interactive conference to a live stream with **millions** of viewers without switching SDKs.
- **Built for Developers, Designed for Scale:** 100ms is built ground up for developers. From elegant APIs, and descriptive error messages, to predictable debug-ability, everything is designed to deploy stable, elegant applications in lesser time to market.
- **Built-in Reliability:** With in-built handling of network and device edge-cases, [and an obsessive focus on reliability](https://www.100ms.live/blog/how-100ms-tests-for-network-reliability), 100ms makes it noticeably easier to achieve higher call completion rates.
- **Quick-Start Starter Kits:** With open-sourced UI components and production-ready templates, getting started with **100ms** takes minutes, not months!

## Feature Comparison - Twilio vs. Agora vs. 100ms

At a high level, here’s how **100ms’ SDK** compares with **Twilio** and **Agora** in terms of features:

|  | Twilio | Agora | 100ms |
| --- | --- | --- | --- |
| Video Calling/Conferencing | ✅ Available. | ✅ Available. | ✅ Available. |
| Live Streaming | Can be built with Twilio Live SDK. | Can be built with Agora Interactive Live Streaming SDK. | ✅ Available. |
| Chat | Can be built with Twilio Conversations API. | Can be built with Agora Chat API (Beta). | ✅ Available. |
| Raise Hand | Possible to implement with DataTrack API without a persistent state. Might not be visible to newly joining peers. | Possible to implement with Agora RTM SDK. To raise a hand, a message with a request can be sent. | Easy to implement with Peer Metadata with a persistent state that is visible even to newly joining peers. |
| Video Background Modification | Can be built with Twilio Video Processor Library. | Can be built with Agora Virtual Background Extension. | ✅ Pre-built virtual background feature available. |
| Screen Share | ✅ Available. | ✅ Available. | ✅ Available. |
| Mute/Unmute Remote Peer | Possible to implement with DataTrack API by sending a message. | Possible to implement with RTM SDK by sending a message. | ✅ Available. |
| Remove (Kick) Remote Peer | ❌ Not available. | ❌ Not available. | ✅ Available. |
| Persistent Peer State | ❌ Not available. | ❌ Not available. | ✅ Available. Known as Peer Metadata. |
| Noise Reduction | ✅ Available. | ✅ Available. | ✅ Available. |
| Recording | ✅ Available. | ✅ Available. | ✅ Available. |
| Platforms/Frameworks | Web, Android, iOS. | Web, Android, iOS, Flutter, React Native, Windows, macOS, Unity, Electron. | Web, Android, iOS, Flutter, React Native. |

In addition to the feature comparison, here are a couple of examples of how 100ms has thought through little things contributing to easier and faster development:

### Difficult to Manage Pub/Sub? Roles to the Rescue

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/100ms_three_roles_efc749e9a4/100ms_three_roles_efc749e9a4.jpeg) 

Managing the publish/subscribe permissions for a peer can be a tedious task. Both **Twilio** and **Agora** SDKs do not offer any out-of-the-box solution, making it even more difficult to design permissions the way you imagine.

**100ms** simplifies this problem with the concept of `Roles`.  `Roles` allow developers to deal with the collection of permissions that a peer has and determine whether a peer has appropriate publish/subscribe rights. For example, `Roles` can signal whether a peer can mute/unmute another peer in the same room. 

You can use pre-defined roles or create your own without any additional coding. This significantly reduce the effort required to manage peer permissions. For a deep-dive into how Roles work, check out this [piece on roles in 100ms](https://www.100ms.live/blog/roles-on-100ms).

### Offer Custom Interactivity (& more) with Peer Metadata

It is possible to store the persistent state of a peer with **100ms**. The `Peer Metadata` can be used to set data that can be updated anytime and is persisted across peers as they join and leave a room. Many useful features like raise hand and storing extra information about a peer are implemented reliably through this offering.

The persistence of this data is a key differentiation with 100ms. To contrast, both Twilio and Agora offer interactive capabilities like the **raise hand** feature (enabling or disabling it). But the way in which they are implemented compromises on reliability at times. For **Twilio**, this can be [implemented](https://www.twilio.com/blog/raise-hand-video-application) with `DataTrack` and for Agora, **[RTM SDK](https://stackoverflow.com/questions/68139244/display-user-name-on-remote-videos-on-group-chatting-raise-hand-feature-on-grou)** has to be used. In both cases, a corresponding message is sent to all the peers in a room. But subsequent peers joining the room will not see this. 

Persistent `Peer Metadata` in 100ms ensures every peer has access to the same interactions - a peer can raise or lower hand, and all other peers in the room will be notified. But this information will also be available for peers joining later to view and consume.

## So, should you migrate from Twilio to 100ms?

Instead of just saying yes, we simply recommend that you try out the [examples we’ve built out with 100ms](https://www.100ms.live/examples). Not enough? Then, try building your own apps with 100ms SDKs easily with concise Quickstart guides — [Web](https://www.100ms.live/docs/javascript/v2/guides/javascript-quickstart), [Android](https://www.100ms.live/docs/android/v2/guides/quickstart), [iOS](https://www.100ms.live/docs/ios/v2/guides/quickstart), [Flutter](https://www.100ms.live/docs/flutter/v2/guides/quickstart) and [React Native](https://www.100ms.live/docs/react-native/v2/guides/quickstart).

If you want to envision how the migration process to 100ms might look like, do check these out:
- [Comparing the Code: Migrating from Twilio to 100ms [Web Edition]](https://www.100ms.live/blog/migrate-twilio-to-100ms)
- [Comparing the Code: Migrating from Agora to 100ms [Web Edition]](https://www.100ms.live/blog/migrate-agora-to-100ms)

why-migrate-twilio-agora-to-100ms-cover.png

large_why-migrate-twilio-agora-to-100ms-cover.png

medium_why-migrate-twilio-agora-to-100ms-cover.png

small_why-migrate-twilio-agora-to-100ms-cover.png

thumbnail_why-migrate-twilio-agora-to-100ms-cover.png

Why Migrate from Twilio/Agora to 100ms?

In 2020, you might’ve caught Dua Lipa’s STUDIO 2054 virtual concert - the one that drew 5 million viewers. Or were you one of the [8.4 million livestream viewers](https://metro.co.uk/2020/06/16/fortnite-device-live-event-broke-internet-12-million-players-12857763/) (and Fortnite fans) who tuned in to watch Epic Games' in-game event "The Device" in 2020? 

Perhaps you bought yourself something from a live shopping event. These events generated almost [US $5.6 billion sales in 2020](https://www.muvi.com/blogs/top-10-live-streaming-use-cases-in-2022.html) so it’s not a stretch to imagine you or someone you know shopping off one. 

Guess what’s common about the three wildly different events we just mentioned? They’re all livestreams. 

Gone are the days where live streams were largely about individual influencers telling you what they had for lunch. It’s value for businesses operating in different industries is now obvious and real-world approved. 

But anecdotes aside, global consumption of live streams seems to be on the rise:

- 80% of audiences prefer watching a live video (livestreams) to reading a blog, based on a survey of 1000 adults spanning different ages and professions. [Source](https://livestream.com/blog/live-video-statistics-livestream#:~:text=80%25%20would%20rather%20watch%20live,in%20second%20place%20with%2043%25.).
- 63% of people aged 18-34 are watching live streamed content regularly. [Source](https://neilpatel.com/blog/live-streaming-importance-2018/).
- 67% of audiences who watched a live stream purchased a ticket to a similar event the next time it occurred. [Source](https://techjury.net/blog/live-streaming-statistics/#gref).
- Live content gets 27% more view minutes per view compared to video on demand (just under 18 minutes). [Source](https://www.conviva.com/state-of-streaming/).

Organizations are [finally starting to catch up on the efficacy of livestreams](https://www.warc.com/newsandopinion/news/us-brands-adopt-e-commerce-livestreaming-to-boost-sales/en-gb/44218) when it comes to generating brand and product awareness as well as revenue. Chances are, if you’re reading this article, your organization has realized the benefits of implementing live streaming into your customer-facing apps & websites.

However, if you are still asking yourself “What is Live streaming?”, this article is a good place to start. It’s your 101 guide to what live streaming is, how it works as well as a quick view of the technical elements live streaming requires.

## What is Live streaming?

Chances are, you’ve already watched at least one (or more realistically, tens or hundreds) live stream by now. Essentially, live streams involve sending video and audio over the internet in near real-time. 

Think of live streams as watching live news or sports. Now extend that thought to include concerts, video games, fashion shows, travel experiences broadcast in real time….

In live streams, video is captured, processed, and delivered to viewers almost simultaneously. 

All viewers need to watch a live stream is an internet-enabled device and access to the streaming platform. But what actually happens between when the moment is “captured” and the moment when it is actually consumed by you?

## How does Live streaming work?

> **Note:** The workflow described below is largely how livestreaming works when implemented via HLS - the most popular (almost default) protocol for video-delivery over the internet. However, there are other livestreaming protocols that may require a different set of steps to push livestreams to viewers.

In order for video to be streamed live, it goes through the following stages:

- Video capture
- Encoding
- Segmentation
- Content delivery network (CDN) distribution
- Decoding & Playback

 ![](https://storage.googleapis.com/100ms-cms-qa/cms/HLS_65d079c846/HLS_65d079c846.png) 

### Video Capture

First, video is captured by a digital camera. Generally, such cameras capture video at about 4k resolution, which is too high to be transported to viewer devices in its raw form. 

Note that all cameras perform some compression automatically on captured images, in order to store them efficiently. However, even after compressing them, the files are too large to be transferred over most internet connections. This is especially true for livestreams catering to large audiences logging in from different locations and different network connectivity levels. 

### Encoding

The raw video captured by the camera cannot be directly transmitted to viewers. Video must be compressed - converted from heavy, raw files to smaller, compressed files. 

Think of your video as a collection of still images, shown one after the other, at a certain pace that is fast enough to create the illusion of movement to the human eye. Encoding packs this sequence of still images efficiently, by eliminating repetition and utilising the commonalities between subsequent frames.

For example, let’s say that the first frame of the video depicts a dancer twirling against a blue wall. If the background does not change for the next few frames, the background does not have to be separately rendered for those subsequent frames. Encoding can potentially remove the data for the background for all frames except the first one - thus reducing the data required to depict that background.

Encoders use ‘codecs’ (short for coder-decoder) to compress video files by removing redundant frames. This converts massive RAW files reasonably-sized units that are easier to stream over most internet connections. 

> **Note**: Codecs are protocols or processes that define how video files will be stored post encoding. The example above of the dancer and the blue wall depicts what codecs do - reduce file size by elimination unnecessary frames while retaining quality as far as possible. Different codecs have different ways of accomplishing. 

It is possible to convert video files from one codec to another during the livestreaming process. However, that is beyond the scope of this article, and will be explored in detail in an upcoming piece.  

### Segmentation

After encoding, video files are directed to a media server. Even though the video files are now more streamable, they still need to be further processed for streaming to users with different devices and internet speeds. For this purpose, the server segments the video and converts these segments into multiple renditions or bitrates. 

To start with, the video files are broken down into chunks or segments. The segment length can be decided beforehand to suit your use-case. For example, you can split a one-hour long video into 360 10-second segments.

Next, each of these segments are converted to different renditions or bitrates. For example, in this case, the 360 segments are converted into:

- 360 segments at 1080p
- 360 segments at 720p
- 360 segments at 360p

Segmentation is required so that users can watch livestreams without interruptions, no matter the strength of their network connection. Modern video players (most of which are HTML5 players) can actually choose and pick which video segments to play, depending on the bandwidth available. It can also automatically switch between playing video segments at different bitrates mid-stream, if the internet strength improves or degrades.

Segmentation is required to achieve something called adaptive bitrate streaming (ABR) - a phenomenon at the heart of modern, democratized livestreams. Keep scrolling, and you’ll find ABR is explained in detail later in this article. 

## CDN distribution and caching

Any livestream catering to even hundreds of users requires the video server to store, queue, and process formidable volumes of data. Moreover, with users joining from multiple devices, a single video server responding to requests is bound to experience load-based stress and even crash. To prevent this from happening, livestreaming stacks use Content Deliver Networks (CDNs).

A CDN is a network of interconnected servers placed across the world. The main criteria for distributing content (video chunks in this context) via CDNs is how physically close the server is to the end-user. Here’s how it works:

Once your viewer clicks the play button, their video players trigger a request for the livestream content. The request is routed to the closest CDN server. In case this is the first time this content is being requested, the server will reroute the request to the original server where the video segments are processes and stored. In response, the origin server sends the video to the CDN server.

The CDN server will forward the video segments to the user. But it will also cache a copy of it locally. When requests for the same content come through again, the CDN does not have to trigger another request to the origin server. It can just forward the cached files. 

The whole point of CDNs is that they are scattered across the world. This helps with reducing latency in livestreams (or any video content) because viewer requests do not have to travel inordinately long distance every time someone wants to watch online content.

## Decoding and playback

All video data reaches the end-user’s player in encoded format, as discussed above. The player decodes and decompresses the video segments in order to play them. This player could be a dedicated app or function within a browser. 

In order to decode the encoded and compressed files, the video player must have a relevant codec already installed. At the very least, it should have the requisite codec installed. 

## Adaptive Bitrate Streaming (ABR)

Adaptive bitrate streaming is a technology that ensures that viewers of online video content get the best possible video quality, as allowed by their bandwidth availability. It is designed to provide viewers with uninterrupted video streams, irrespective of changes in a user’s internet speed. 

As explained above, during segmentation, video chunks are processed into different bitrates. Depending on the end-users’ internet strength, the video player chooses the rendition that would be best assured to provide swift buffering and seamless playback. 

For example, let’s say you’re watching a livestream via your home wi-fi connection. Chances are, you connection is high-speed, which means that you’ll probably get video segment at 720p or 1080p. However, if you decide to watch the stream at a cafe while waiting for your coffee order, you might hit a spot of bad connection, which is when your player will select and deliver video segments at 360p or lower. That way, the video never pauses and you never miss out on the content; you just might have to watch it at a lower quality. 

## Interactivity for Greater Audience Engagement

Livestreams allows the opportunity for real-time interaction with audiences. Most live streams at least allow for a chat function, through which users can leave comments and ask questions. 

However, livestreams are now moving beyond basic text-based interactivity. Audience now expect to be seen and heard interacting with the streamers. For example, it is common for celebrities on Instagram to host live videos in which they select members from the audience to talk to, while the rest of the viewers watch. 

Also, since livestreams too can now be recorded, they can be reupload as on-demand content for viewers who may not have caught the stream in real-time. Essentially, live streams are now able to target multiple viewer segments. 

Bear in mind that users also do not simply want to sit and be spoken to. They want to engage, highlight what matters to them, ask questions, have their opinions heard. Livestreams must support this expectation via features like polls, chat, video interaction & more - all of which are supported with 100ms.

The use-cases for interactive livestreams are also expanding into industries you wouldn’t imagine would align with this technology. For example, live shopping allows sellers to showcase their products online, and lets customers buy said products in real-time. This phenomenon has exploded in China, to the extent that [it is being hailed as the future of shopping](https://www.bloomberg.com/news/videos/2021-06-25/the-future-of-shopping-is-already-happening-in-china-video). 

## Build & Run Interactive Livestreams with a single SDK

If you’re planning on adding livestreams to your video content strategy, it's recommended that you build with interactivity in mind. 100ms enables exactly this by offering a single SDK which combines streaming with conferencing. Our [Interactive Live Streaming](https://www.100ms.live/interactive-live-streaming) SDK enables customers to build more engaging live streams with multiple broadcasters, interactivity between broadcaster and viewer, and the ability to easily stream from mobile.

At a high level, the 100ms SDK lets you build two-way interactive live streams into your product with minimal effort. Broadcasters can go live from any browser, and invite audience members to the stage in order to communicate with them directly, on video. We’ve also  built primitives which allows for building varied interactions (chat, polls, call metadata) on top of our SDK.

Additionally, you get to customize your screen layout, and launch streams directly from apps or desktop browsers without having to use professional stream encoding software of any kind. 

[Get Started for Free](https://www.100ms.live/interactive-live-streaming) 

If you’re not convinced, just click on the link above and try out the SDK for free.