TECHNICAL FIELD
[0002] The present disclosure is directed to encryption of a data stream, and more
specifically, pulse-code modulation (PCM) audio data, such as PCM audio related to live,
streaming audio.
BACKGROUND
[0003] Digital and PCM audio are frequently used in streaming audio contexts. The
streaming audio data is frequently streamed in 12, 16, or 24-bit variations. Some
restrictions apply to streamed audio, such as the inability to add data, and the ability to
throw away some of the data. In some circumstances the packet frame size is also fixed
(e.g., 64, 128, or 256 bit).
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] Figure 1 is a block diagram illustrating an overview of devices on which some
implementations of the present technology can operate.
[0005] Figure 2 is a block diagram illustrating an overview of an environment in
which some implementations of the present technology can operate.
[0006] Figure 3 is a block diagram illustrating components which, in some
implementations, can be used in a system employing the disclosed technology.
[0007] Figure 4 is a flow diagram illustrating a process used in some
implementations of the present technology for encrypting live data streams.
[0008]
[0009]
Figure 5 is block illustration of data stream packets.
Figures 6A-F are block illustrations of various steps of some implementations
of the disclosed protocol wherein:
wo 2022/066662 PCT/US2021/051352
[0010]
[0011]
[0012]
Figure 6A is a block diagram of a set of samples;
Figure 68 is a block diagram of the set of samples after truncation;
Figure 6C is a block diagram of the truncated set of samples superimposed
over the space required for the samples of Figure 6A;
[0013] Figure 6D is a block diagram of the truncated set of samples combined with
encryption data representing a data stream unit;
[0014] Figure 6E is a block diagram illustrating the data stream unit packed into 128-
bit data frames; and
[0015]
protocol.
[0016]
Figure 6F is a block diagram illustrating structure of an HMAC authentication
The technology and techniques introduced here may be better understood
by referring to the following Detailed Description in conjunction with the accompanying
drawings, in which like reference numerals indicate identical or functionally similar
elements.
DETAILED DESCRIPTION
[0017] Aspects of the present disclosure are directed to encryption of data streams.
Some embodiments pertain to digital or PCM live streaming audio data. Raw streamed
audio data is not encrypted and may be vulnerable to attacks on compromised devices.
Common protocols for encryption make use of 128-bit frames (though other frame sizes
are feasible). The size is based on common ciphers, such as Advanced Encryption
Standard (AES) that process 128-bits at time. PCM audio is typically not encrypted and
may be boundaryless (e.g., there is no need for an indication of where in an audio stream
the receiver is). However, when PCM audio is encrypted, the audio must necessarily be
decrypted to be played. Decryption of the PCM audio must be precise with respect to
sample boundaries or the data will not decrypt correctly. The transmitted audio data
stream is received by a receiver, although the receiver is not able to automatically
determine the start of the 128-bit boundary so some sort of signaling must be used.
[0018] Disclosed herein is a protocol to encrypt audio over an existing digital audio
link (ex: Audinate Dante®, AES-3, AES-67 or QSC Q-LAN™). Audio interfaces, like AES-
3, do not include any encryption features. One cannot increase the number of bits in a
-2-
wo 2022/066662 PCT/US2021/051352
stream; however, by reducing the bit depth of the audio data (e.g., from 24 to 22, or 20),
one makes space for new data. By reducing each audio sample by 2 bits, enough bits
are conserved for a frame of new, encryption data every 12th frame. The encryption data
frame includes a counter portion and a Hash-based Message Authentication Code
(HMAC) signature. The counter portion is employed as a public nonce while the HMAC
is used to verify the boundaries of each 128-bit frame and authenticate the data in the
frame.
[0019] Several implementations of the present technology are discussed below in
more detail in reference to the figures. Figure 1 is a block diagram illustrating an overview
of devices on which some implementations of the disclosed technology can operate. The
devices can comprise hardware components of a device 1 00 that transparently encrypts
audio data. Device 1 00 can include one or more input devices 120, including
microphones, that provide input to the Processor(s) 110 (e.g. CPU(s), GPU(s), HPU(s),
etc.), notifying it of actions. The actions can be mediated by a hardware controller that
interprets the signals received from the input device and communicates the information
to the processors 11 0 using a communication protocol. Input devices 120 include, for
example, a mouse, a keyboard, a touchscreen, a sensor (e.g., an infrared sensor), a
touchpad, a wearable input device, a camera- or image-based input device, a
microphone, or other user input devices.
[0020] Processors 110 can be a single processing unit or multiple processing units
in a device or distributed across multiple devices. Processors 110 can be coupled to
other hardware devices, for example, with the use of a bus, such as a PCI bus or SCSI
bus. The processors 110 can communicate with a hardware controller for devices, such
as for a display 130. Display 130 can be used to display text and graphics. In some
implementations, display 130 provides graphical and textual visual feedback to a user.
In some implementations, display 130 includes the input device as part of the display,
such as when the input device is a touchscreen or is equipped with an eye direction
monitoring system. In some implementations, the display is separate from the input
device. Examples of display devices are: an LCD display screen, an LED display screen,
a projected, holographic, or augmented reality display (such as a heads-up display device
or a head-mounted device), and so on. Other 1/0 devices 140 can also be coupled to
the processor, such as a network card, video card, audio card, USB, firewire or other
wo 2022/066662 PCT/US2021/051352
external device, camera, printer, speakers, CD-ROM drive, DVD drive, disk drive, or BluRay
device.
[0021] In some implementations, the device 1 00 also includes a communication
device capable of communicating wirelessly or wire-based with a network node. The
communication device can communicate with another device or a server through a
network using, for example, TCP/IP protocols. Device 100 can utilize the communication
device to distribute operations across multiple network devices.
[0022] The processors 11 0 can have access to a memory 150 in a device or
distributed across multiple devices. A memory includes one or more of various hardware
devices for volatile and non-volatile storage and can include both read-only and writable
memory. For example, a memory can comprise random access memory (RAM), various
caches, CPU registers, read-only memory (ROM), and writable non-volatile memory,
such as flash memory, hard drives, floppy disks, COs, DVDs, magnetic storage devices,
tape drives, and so forth. A memory is not a propagating signal divorced from underlying
hardware; a memory is thus non-transitory. Memory 150 can include program memory
160 that stores programs and software, such as an operating system 162, encryption
module 164, and other application programs 166. Memory 150 can also include data
memory 170, e.g., the PCM streaming audio data, configuration data, settings, user
options or preferences, etc., which can be provided to the program memory 160 or any
element of the device 1 00.

CLAIMS
1 . A method comprising:
reducing a predetermined bit resolution of audio samples included in an audio
stream, resulting in a set of reduced bit resolution audio samples, wherein the audio
stream is configured for transmission at a predetermined frame size;
encrypting the set of reduced bit resolution audio samples;
supplementing the set of encrypted reduced bit resolution audio samples with
encryption data frames, the encryption data frames occurring at a frequency in a modified
audio stream based on a number of bits reduced from each audio sample and the
predetermined frame size, the modified audio stream including the set of encrypted
reduced bit resolution audio samples and the encryption data frames; and
transmitting the modified audio stream.
2. The method of claim 1, further comprising:
in response to said transmitting, receiving modified audio stream including the set
of encrypted reduced bit resolution audio samples and the encryption data frames;
synchronizing the set of encrypted reduced bit resolution audio samples based on
the encryption data frames;
decrypting the set of encrypted reduced bit resolution audio samples;
restoring the bit resolution of the set of reduced bit resolution audio samples using
generic bits matching a number of bits reduced from the audio samples, resulting in an
audio stream including audio samples with the generic bits and having the predetermined
bit resolution; and
passing the audio stream including the audio samples with the generic bits and
having the predetermined bit resolution to a receiver audio interface.
3. The method of claim 1, wherein the encryption data frames include a Hashbased
Message Authentication Code (HMAC) signature portion and a counter portion.
4. The method of claim 1 ,
wherein the predetermined frame size is 128 bits, the predetermined bit resolution
is 24-bit, and the number of bits reduced from each audio sample is 2 bits; and
-14-
wo 2022/066662 PCT/US2021/051352
wherein the frequency of that the encryption data frames occur is every twelfth
frame.
5. The method of claim 1, wherein said reducing the predetermined bit
resolution comprises:
truncating a predetermined number of bits from the end of each audio sample.
6. The method of claim 5, further comprising:
determining a latency based on the predetermined number of bits truncated from
the end of each audio sample.
7. The method of claim 1, wherein each step of the method is performed justin-
time.
8. The method of claim 1, further comprising:
exchanging encryption keys between a transmitter and a receiver of the audio
stream.
to:
9. A system comprising:
a processor;
an audio stream input interface configured to receive an audio stream;
a communication interface configured to transmit a modified audio stream; and
a memory including protocol instructions that when executed cause the processor
reduce a predetermined bit resolution of a set of audio samples included in
the audio stream, the reduction resulting in a set of reduced bit resolution audio
samples, wherein the audio stream is configured for transmission at a
predetermined frame size;
encrypt the set of reduced bit resolution audio samples; and
augment the set of encrypted reduced bit resolution audio samples with
encryption data frames, the encryption data frames occurring at a frequency in the
reduced bit resolution audio stream based on a number of bits reduced from each
audio sample and the predetermined frame size, the augmentation of the set of
-·~ 5-
wo 2022/066662 PCT/US2021/051352
reduced bit resolution audio samples resulting in the modified audio stream
including the set of encrypted reduced bit resolution audio samples and the
encryption data frames.
10. The system of claim 9, further comprising:
an audio stream receiver interface configured to receive the transmitted modified
audio stream including the set of encrypted reduced bit resolution audio samples and the
encryption data frames and synchronize the set of encrypted reduced bit resolution audio
samples based on the encryption data frames,
wherein the audio stream receiver interface is further configured to decrypt the set
of encrypted reduced bit resolution audio samples and restore the bit resolution of the
reduced bit resolution audio samples using generic bits matching a number of bits
reduced from the audio samples, the restoration resulting in an audio stream including
audio samples with generic bits and having the predetermined bit resolution; and
wherein the audio stream receiver interface is further configured for playback of
the audio stream including the audio samples with generic bits and having the
predetermined bit resolution.
11. The system of claim 9, wherein the encryption data frames include a Hashbased
Message Authentication Code (HMAC) signature portion and a counter portion.
12. The system of claim 9,
wherein the predetermined frame size is 128 bits, the predetermined bit resolution
is 24-bit, and the number of bits reduced from each audio sample is 2 bits; and
wherein the frequency of that the encryption data frames occur is every twelfth
frame.
13. The system of claim 9, wherein said reduction of the predetermined bit
resolution comprises:
truncation of a predetermined number of bits from the end of each audio sample.
14. The system of claim 9, wherein the protocol instructions are executed by
the processor just-in-time.
-Hiwo
2022/066662 PCT/US2021/051352
15. A method of operating an audio stream encryption protocol comprising:
reducing, by an audio stream input interface, a predetermined bit resolution of
audio samples included in an audio stream via truncation of a predetermined number of
bits from the end of each audio sample, resulting in a set of reduced bit resolution audio
samples, wherein the audio stream is configured for transmission at a predetermined
frame size;
encrypting, by the audio stream input interface, the set of reduced bit resolution
audio samples, said encryption including augmentation of the set of reduced bit resolution
audio samples with encryption data frames, the encryption data frames occurring at a
frequency in a modified audio stream based on a number of bits reduced from each audio
sample and the predetermined frame size, the modified audio stream including the set of
encrypted reduced bit resolution audio samples and the encryption data frames;
transmitting, by the audio stream input interface to an audio stream receiver
interface, the modified audio stream;
synchronizing, by the audio stream receiver interface, the set of encrypted
reduced bit resolution audio samples based on the encryption data frames;
decrypting, by the audio stream receiver interface, the set of encrypted reduced
bit resolution audio samples;
in response to said decrypting, restoring, by the audio stream receiver interface,
the bit resolution of the set of reduced bit resolution audio samples using generic bits
matching the predetermined number of bits, resulting in an audio stream including audio
samples with generic bits and having the predetermined bit resolution; and
playing, by the audio stream receiver interface, the audio stream including audio
samples with generic bits and having the predetermined bit resolution.