NVIDIA Broadcast — our tool for livestreaming and video conferencing that turns your room into a home studio — is updating today with two exciting effects, Eye Contact and Vignette, drastic enhancements to Virtual Background and more.
You can now use Credential Control Properties to more easily restrict the usage of your IAM Roles for EC2.
In recent months, satellite services for smartphones have been hyped a lot. While surprising for many, including me, this didn’t come out of nowhere, and a lot of companies have worked on this topic for quite some time. 3GPP has also picked up the topic a few years ago with study items in 3GPP Release 15 and 16. Now, with Release 17, the LTE and 5G NR air interfaces have been extended for use over satellites. The term for this in 3GPP: Non-Terrestrial Networks (NTN). So let’s have a look at what has actually been specified:
Both the LTE and 5G NR air interfaces have been designed and optimized to transfer data over relatively short distances of a few kilometers or a few tens of kilometers in extraordinary circumstances. When communicating with Low Earth Orbit (LEO) satellites, distances that need to be covered are well beyond 800 km, and communicating with geostationary satellites requires data to be sent over a distance of 35.000+ km. With NTN, 3GPP adds a few bits and pieces to use the LTE and 5G NR air interface to and from satellites. In fact, 3GPP has looked at two very different scenarios:
Stationary Mobile Terminals with Parabolic Antennas
The first scenario describes how to use the LTE/5G NR air interface with stationary terminals and a parabolic antenna for high data rate applications. Starlink, Amazon’s Kuiper and OneWeb are examples of this, even though I suppose all of them use a proprietary protocol and data transmission method. I can’t be sure however, as there is very little technical documentation available.
Mobile Satellite Services
Secondly, 3GPP has looked at how data could be sent to and from mobile devices with small and non-directional antennas. For this scenario, even the most optimistic people in 3GPP don’t see high speed Internet access to be delivered to mobile devices from space anytime soon. As the 5G air interface is currently only used for high speed applications, 3GPP has also taken a step back and extended the LTE specification for NTN, particularly for the two bandwidth and speed reduced flavors: LTE NB-IoT and LTE CAT-M.
NTN in 3GPP
For both scenarios, i.e. high data rate transmission to and from devices with a large dish antenna and low data rate transmissions to and from small mobile devices with built-in antennas, the air interface has only been extended very slightly to address the following differences between short range and long range communication:
Timing Advance: During the random access procedure, the network tells the mobile device how much to time-shift uplink transmissions compared to the observed downlink transmissions to stay in sync. The current parameter only covers a range a few kilometers and hence an addition is necessary to signal larger delays, i.e. longer distances.
HARQ over larger distances: This is a LTE/5G mechanism used to detect and retransmit faulty data blocks that also works with specific timing constraints. Due to longer signal delays, this mechanism needs to be adapted.
Satellite Location Information: For mobile devices to estimate when satellites might become available for communication, details on the satellite constellation and location of all satellites is very helpful. This is known as ‘ephemeris information’, which is added to periodic system information broadcasts.
And that’s pretty much it. ShareTechnote has a great overview of how the air interface protocol for LTE, NB-IoT, CAT-M1 and 5G NR has been extended, so have a look for the details. Tip: Use the web browser’s search function to highlight all occurrences of ‘r17’. Here are some examples of how LTE and NB-IoT were extended:
On the 5G NR side, SIB-19 has been introduced to broadcast NTN configuration information to ground based devices and similar mechanisms like for LTE have likely been specified for the timing advance and the HARQ mechanism to account for the larger delay.
Bent Pipe or eNB/gNBs in Space
Another interesting resource I came across on the web is this article on Microwaves and RF. The article describes that 3GPP has looked at two different ways of using satellites. The first approach, which the article says is what 3GPP Rel. 17 mainly uses, is ‘NTN transparent mode’. Here, the satellite acts as a repeater in both directions. The eNB or gNB, i.e. the traditional LTE/5G NR radio network elements are installed on the ground. Such a setup is also referred to as a ‘bent-pipe’ approach. A current (non-3GPP) satellite constellation that uses this approach is Globalstar.
The second approach is for the satellite to act as an eNB or gNB in space. While more complex, the advantage of such an approach is data can be forwarded between satellites when a ground station is not in view. This approach is used, for example, by Iridium since the 1990’s, and only 3 ground stations are in operation around the globe. Satellites communicating with each other is not science-fiction, but science fact for a long time now.
Frequency Ranges
The article also gives some details on the frequency ranges discussed in 3GPP for NTN which are perhaps globally available: The first range of 30 MHz is between 1980 to 2010 in the uplink direction and 2170 to 2200 MHz in the downlink direction. And then there’s 34 MHz between 1626.5 and 1660.5 MHz for the uplink and between 1525 MHz to 1559 MHz for the downlink. Again a comparison: Iridium uses a 10 MHz chunk in the 1.6 GHz range, so the general frequency ranges that are looked at by 3GPP have been in use for satellite communication for a long time and hence, characteristics are well known.
Compared to what is used by terrestrial LTE and 5G networks, two times 30 MHz is not much. I guess which spectrum demand below 6 GHz being so high for terrestrial services, it’s unlikely that more could be allocated to satellite services, which will of course limit single user speeds and overall network capacity.
And finally, 3GPP has also looked at the feasibility of using spectrum at 20 and 30 GHz for NTN. I don’t know how well that would work to and from small mobile devices, perhaps this spectrum could mostly be used for devices with parabolic antennas? Just a guess, if you know more, please consider leaving a comment.
Why Now?
The question I am asking myself is why there is such a hype now? Or in other words: How has the cost equation changed since satellite constellations like Iridium and Globalstar have been launched in the 1990s, which, by the way, are still in use today for IoT-like applications? I guess that apart from the tremendous advances that have been made on computing power as well on electronic and RF components in the last three decades, shooting satellites to space has also become a lot cheaper. Starlink has well over 2000 satellites in orbit today, and more are launched in the order of once a week, 60 of them on a single rocket. Back in the 1990s, that was about as much science fiction as the gigabit datarates terrestrial LTE and 5G networks can deliver today.
Keep in Mind: It’s ‘Narrow-Band’ to Mobile Devices!
Before I close, I think its important to set things into perspective again. Despite all advances, we are not talking about individual datarates of gigabits or even megabits per second to and from mobile devices with small antennas and satellites. This is why 3GPP has focused on LTE NB-IoT and LTE-M for NTN and in the next post on this topic, I’ll have a look again why those narrowband technologies as they are today are so suitable not only for low power and low data rate scenarios with terrestrial networks, but also for satellite communication.
I wrote about forking Chrome to turn HTML to SVG two months ago, today we're going to do something similar by making it render into a terminal.
Let me introduce you to the Carbonyl web browser!
When you think of animating CSS properties, which ones come to mind? I recently started wondering about the ones that don’t come to mind — properties that aren’t typically associated with animation, but turn out to be animatable.
This articles explores some of the unexpected things that CSS can animate and some nifty things you can do by animating them.
When your browser animates a CSS property, it creates a sequence of intermediate values over the course of the animation. This is referred to as interpolation.
Here’s a widget that will animate height
on a <div>
and show the interpolated values during the animation. Give it a try.
See the Pen Interpolation Widget (height) by Will Boyd (@lonekorean) on CodePen.
The results are pretty much what you’d expect. Interpolating height
values from 80px
to 160px
is straightfoward: it’s a series of incremented values.
Now let’s do z-index
.
See the Pen Interpolation Widget (z-index) by Will Boyd (@lonekorean) on CodePen.
Yes, you can animate z-index
! It’s not visually apparent in this demo, but seeing the interpolated values go from 1
to 5
confirms it. What’s interesting is that we don’t see any decimal points like we did for height
. We only get integers, which makes sense for z-index
.
Alright, let’s do something wacky. What happens if we try to animate text-transform
from lowercase
to uppercase
?
See the Pen Interpolation Widget (text-transform) by Will Boyd (@lonekorean) on CodePen.
Something happened! Unless you’re using Safari (sorry). But for everyone else, you can see the text flip from “div” to “DIV”. This is known as a discrete animation and the W3C Web Animations working draft describes the behavior as follows:
The property’s values cannot be meaningfully combined, thus it is not additive and interpolation swaps from Va to Vb at 50% (p=0.5)…
In other words, if there’s not a sensible way to gradually interpolate from the start value to the end value, then just switch between them halfway through.
And there’s our spark. If we look beyond properties with obvious gradual visual transitions, we’ll find that a lot of unexpected properties can be animated, either discretely or with unique interpolations.
Let’s go back to z-index
. You can animate z-index
to achieve layered animations. Here’s an example with Big Boo floating through walls.
See the Pen Spooky Boo! by Will Boyd (@lonekorean) on CodePen.
There are 5 walls, each with a z-index
of 1
through 5
. The Big Boo has its z-index
animated from 1
to 6
, moving it in front of each wall, before alternating back to move behind each wall.
.boo-move {
animation: move 6s ease-in-out infinite alternate;
}@keyframes move {
from {
z-index: 1;
/* `transform` here to appear further away */
}
to {
z-index: 6;
/* `transform` here to appear closer */
}
}
There are additional transform
declarations and a separate bob
animation to complete the effect of the Big Boo floating hauntingly towards us as it goes through walls. Check the CSS in the demo if you’re interested.
Here’s another demo that plays with z-index
, this time to animate overlapping cards.
See the Pen Overlapping Sushi Cards by Will Boyd (@lonekorean) on CodePen.
To swing a card forwards, we apply this animation.
@keyframes swing-forwards {
0% {
z-index: 1;
/* `transform` here to appear further away */
}
50% {
/* `transform` here to swing outwards */
}
100% {
z-index: 2;
/* `transform` here to overlap inwards and appear closer */
}
}
The z-index
starts at 1
(behind) and ends at 2
(in front).
At the same time, the other card swings backwards with this animation. It’s essentially the same, but reversed.
@keyframes swing-backwards {
0% {
z-index: 2;
/* `transform` here to overlap inwards and appear closer */
}
50% {
/* `transform` here to swing outwards */
}
100% {
z-index: 1;
/* `transform` here to appear further away */
}
}
In both cases, the z-index
value flips right in the middle, at 50%
. This is the moment when the cards are not overlapping, so changing the z-index
won’t cause a card to clip through the other.
Here’s a fun one. This demo is an auto-incrementing binary counter with decimal output using no JavaScript. Works everywhere but Safari. And no, I’m not cheesing that decimal output by hardcoding 0
through 15
somewhere.
See the Pen CSS Binary Counter by Will Boyd (@lonekorean) on CodePen.
The demo works by (discretely) animating 2 properties: content
and counter-increment
. Here’s the relevant CSS.
.widget-wrapper {
counter-reset: sum;
}.digit-4 { --increment: 8; }
.digit-3 { --increment: 4; }
.digit-2 { --increment: 2; }
.digit-1 { --increment: 1; }.digit::before {
content: '0';
animation: flip-digit step-end infinite;
animation-duration: calc(var(--increment) * 2s);
}@keyframes flip-digit {
50% {
content: '1';
counter-increment: sum var(--increment);
}
}.decimal-output::before {
content: counter(sum);
}
Each digit has --increment
set with its value in decimal. Conveniently, this value is also used to calculate the digit’s animation-duration
. For example, .digit-4
has an --increment
value of 8
so its animation-duration
works out to 16s
. Its content
property is animated to be '0'
for half that time, then flip to '1'
for the other half, repeating infinitely.
The decimal output uses a CSS counter, a neat feature of CSS that lets you keep a running tally. A sum
counter is started on .widget-wrapper
, then whenever a digit’s content
is animated to '1'
, we also animate counter-increment
to add its --increment
value to sum
. The resulting total of sum
is displayed in .decimal-output
.
What’s neat is that .decimal-output
isn’t actually animated. It’s simply reflecting the result, in real-time, of various counter-increment
properties being animated.
Consider a modal that you want to fade out when dismissed. Animating opacity
takes care of that. But even when an element reaches opacity: 0
it’s still there, invisibly covering page content and blocking clicks.
You could use a bit of JavaScript with setTimeout
to get rid of the modal at the right time. Another option is to (discretely) animate visibility
.
.modal {
animation: dismiss 1s forwards;
}@keyframes dismiss {
to {
opacity: 0;
visibility: hidden;
}
}
And here it is in action.
See the Pen Fading Modal by Will Boyd (@lonekorean) on CodePen.
But wait a second… if visibility
is animated discretely, shouldn’t it flip to hidden
halfway through the animation, before the fade has finished? Normally yes, but visibility
has special interpolation rules!
For the visibility property, visible is interpolated as a discrete step where values of p between 0 and 1 map to visible and other values of p map to the closer endpoint…
In other words, visibility
won’t flip to hidden
until the very last moment. Here’s that interpolation widget again so you can see for yourself.
See the Pen Interpolation Widget (visibility) by Will Boyd (@lonekorean) on CodePen.
Here’s a widget that lets users click once to select all text, then click again to select some of the text. This is something I wrote about in my previous article, so check there if you want more context. Doesn’t work in Safari, but degrades reasonably well.
See the Pen Select All... Then Select Some by Will Boyd (@lonekorean) on CodePen.
The key piece is a discrete animation that changes user-select
from all
to text
after the element containing the text is focused.
code {
user-select: all;
}code:focus {
animation: select 100ms step-end forwards;
}@keyframes select {
to { user-select: text; }
}
Note that the <code>
element needs to be made focusable with tabindex
for this to work.
<code tabindex="0">code snippet goes here</code>
The box-shadow
property can take multiple sets of values, as a comma-separated list, to create multiple shadows. Here’s some CSS we’ll be using soon.
div {
box-shadow:
/* in order: offset-x, offset-y, blur-radius, color */
0 0 20px #fff,
-20px 0 80px #f0f,
20px 0 80px #0ff, /* same, except these shadows go inwards */
inset 0 0 50px #fff,
inset 50px 0 80px #f0f,
inset -50px 0 80px #0ff,
inset 50px 0 300px #f0f,
inset -50px 0 300px #0ff;
}
By animating box-shadow
, you animate a list of shadows into another list of shadows. Each shadow in the starting list is interpolated to the corresponding shadow in the ending list. Each interpolation is smooth, unless one has inset
and the other doesn’t, in which case the entire animation falls back to being discrete.
Let’s add the following CSS to animate the box-shadow
from earlier. This will animate the offset-x
values of some of the shadows.
div {
animation: pulsate 6s linear infinite;
}@keyframes pulsate {
50% {
box-shadow:
0 0 20px #fff,
20px 0 80px #f0f,
-20px 0 80px #0ff,
inset 0 0 50px #fff,
inset -50px 0 80px #f0f,
inset 50px 0 80px #0ff,
inset -50px 0 300px #f0f,
inset 50px 0 300px #0ff;
}
}
The result is a soft pulsating effect, with colored shadows shifting side to side.
See the Pen Pulsating Marble by Will Boyd (@lonekorean) on CodePen.
The same concept applies to text-shadow
. Hover/tap that chicken nugget to see another animation using multiple shadows.
See the Pen That Chicken Nugget by Will Boyd (@lonekorean) on CodePen.
The HTML wraps each word in its own <span>
.
<div class="words">
<span>look</span>
<span>at</span>
<span>that</span>
<span>chicken</span>
<span>nugget</span>
</div>
And here’s the relevant CSS.
.words span {
text-shadow: 5px 5px 0 #ccc;
}/* add animation to each word when nugget is hovered */
.nugget:hover ~ .words span {
animation-name: pop-word;
animation-duration: 1s;
animation-fill-mode: forwards;
}@keyframes pop-word {
to {
text-shadow:
5px 5px 0 yellow,
10px 10px 0 orange,
15px 15px 0 red,
20px 20px 0 brown;
}
}/* wait half a second between each word */
.words span:nth-child(2) { animation-delay: 0.5s; }
.words span:nth-child(3) { animation-delay: 1s; }
.words span:nth-child(4) { animation-delay: 1.5s; }
.words span:nth-child(5) { animation-delay: 2s; }
Notice how it animates a single gray shadow to multiple colored shadows. In these cases where the starting and ending lists aren’t the same length, the shorter list is padded with “null” shadows (transparent and lengthless) so the lists match up nicely for interpolation.
Hopefully I’ve shown you some interesting things that can be achieved by animating CSS properties not often considered for animation. There are many more properties to play with that this article didn’t get to, so keep exploring!
My intention was to show what’s possible, but as always, please use your own judgement before applying any techniques. Not everything needs to be animated and sometimes JavaScript is the right answer.
By Gerardo Delgado on January 12, 2023 |
NVIDIA Broadcast — our tool for livestreaming and video conferencing that turns your room into a home studio — is updating today with two exciting effects, Eye Contact and Vignette, drastic enhancements to Virtual Background and more.
Broadcast continues its strong momentum with double the active users from the previous year. And over 20 partners have integrated Broadcast effects directly into their apps. Broadcast is also preinstalled in select laptops from Acer, Asus, Gigabyte, MSI, Lenovo and Razer providing users superior, AI-powered experiences with their microphone, webcam and speakers.
The updated Virtual Background effects are also available directly in your favorite apps including OBS, Streamlabs and Elgato Camera Hub. Get the enhanced effects by updating to the latest version of the redistributables in the NVIDIA Broadcast Download Center.
Get started and download NVIDIA Broadcast today, free for all NVIDIA and GeForce RTX GPU owners.
The new Eye Contact effect moves the eyes of the speaker to simulate eye contact with the camera — achieved by estimating and aligning gaze. The eyes retain their natural color and blinks, and there’s even a disconnect feature in case you look too far away, to transition smoothly between simulated and real eyes.
Eye Contact is ideal for content creators seeking to record themselves while reading their notes or a script, or avoiding having to stare directly at the camera. Video conference presenters will look at the eyes of the person they’re speaking with, improving engagement with their audiences.
Currently in beta, we want to work with you, the community, to help improve Eye Contact. There are millions of eye colors and lighting combinations. If you test it and find any issues, or just want to help us develop this AI effect further, please send us a quick video here, we would really appreciate it!
A Vignette effect - similar to those seen in apps like Instagram - has been integrated into Broadcast. Combine it with a subtle Background Blur effect to get an AI-simulated bokeh visual on your webcam, instantly improving visual quality.
In addition, the Blur, Replacement, and Removal Virtual Background effects - have been enhanced with temporal information to achieve better segmentation and stability, enabling the AI to make better decisions. In cases where the network makes a mistake, you will not see parts of the background popping in and out as much.
Any Virtual Background issues? Please send us a video (link) so we can improve the effect for you.
Broadcast has also integrated two of the most popular community requested features: the option to mirror your camera, and to take a screenshot of your webcam! Check out the two new icons under the camera preview labeled Camera Mirroring and Selfies.
Want more features and effects? Continue to share your concerns, report bugs or make requests in the forums. Your feedback is incredibly valuable to the entire team, we can’t stress that enough.
App developers can integrate the SDKs powering NVIDIA Broadcast - known as Maxine - into their apps, including the latest features. At CES 2023, NVIDIA Maxine announced SDK updates, including the new Eye Contact feature; improved performance and quality for Virtual Background; a Speaker Focus feature; will also be available as microservices; and opened early access to test out our latest feature: Live Portrait.
Take your livestreams, voice chats, and video conference calls to the next level with AI-enhanced voice and video with Broadcast.