NeuraWatermark : Deep CNN Image Watermarking for Traceable
Copyright Protection
Field of Invention
~ ‘
The invention pertains to the field ofdigital image processing and deep learning. It involves
a novel approach to watermarking by utilizing Convolutional Neural Networks (CNNs) for both
watermark embedding and extraction. This technique enhances watermark robustness and
imperceptibility, addressing limitations of conventional methods. The innovation lies in the
integration ofCNNs, significantly improving the security and fidelity ofwatermarked images‘ The
invention finds applicability in various sectors, including content protection, copyright
enforcement, and digital asset management.
Background of the invention and the grior art
The invention of deep leaming-based digital image watermarking using:’ a CNN algorithm
represents a significant advancement in the field ofdata security and copyright protection for digixal
content. Prior to this invention, traditional walermarking techniques relied on simplistic methods
that often resulted in easily removable watennarks or compromised image quality. These
Igchniques were insufficient in addressing the groQing need'for robust and imperceptible
waterrfiarking solutions. Moreover, the surge in deep learning approaches, particularly
Convolutional Neural Networks (CNNs), offered promising potential for addressing these
shortcomings by leveraging their capacity to learn intricate image features and panems,
The prior art in digital image watermarking mainly encompassed conventional methods
such as frequency domain techniques and spatial domain xechniques: like discrete cosine transform (DCT) and discrete wavelet transform (DWT) based methods. These lechniqhes were typically
shallow, lacking the ability to adapt to complex variations in image content and often suffering
from low robustness against various attacks.The utilization ofCNNs for watermarking purposes
was a novel direction in the field, as CNNs exhibited the potential to capture intricate spatial
relationships in images and ‘0 automatically learn relevant features for walennark embedding and
extraction. This invention thus represents a paradigm shifi= combining the power of deep learning
with digital walemarking to achieve enhanced robustness and imptceptibility in protecting digital
content from unauthorized use and manipulation.
The invention disclosed in patent numbered US$699089B2 has proposed an Image walemmrking technique lhal involves laking RGB image data and transforming it into CMYK data
‘Ihal defines a "conslam K" image. Then. you gel IR mark data lhal defines an IR mark. and you
define a 2D black pixel pattern that matches the IR data. like Gold codes or random binary
sequences. You then embed the 2D pattern ifilo the CM YK black separalion K, while keeping the
local average levels the same. When you‘re done‘ you have a watermarked image that includes the
2D blahk separation K. plus all the other separations from the CMY image. To do this, you use an
IR microécope that has IR LEDS‘and IR pass filters to take an IR image. Afier that, you do some
exposure correction. thinnings. and diligences to extract the 2D panem for later correlation
processing.
The 1m eniim‘. disclogd i1}. patem numbe-red KR'I 020060001107A relaxes [0 a lechnique
for digilai \x'atermarking that employs size imurian: Emure u‘ansfomwrion. and in particular.
additional \\'alemmrking applied [0 circular paléhcs Kim: are robusl l0 geometric lransibrmalion and
are extracted through. lhis technique. ll concerns a techniqueThe insenion step and the deleclion
stage make L'lp the majority of the digital watermarking process using the size invariant feature
lrunsfomwlion oflhe present invemioa the watermark smbedding step. a circular patch is created
using SHIF. a 2D rectangular watermark with a Gaussian distribution is created using a random
number generator. and the rectangular walermark's invent-d polarizalion is Changed into a circular
watermark based on the size of the circular paxch. The watermark detection process also creates a
circle
The-"invention disclosed in patent numbered US$023157B2 a‘ rear projection picture is
removed by an image data cortection 1001 without removing the image that corresponds to a lowimensity
halfione image‘ The difference in intensity between first image data repi'esenting a portion
of a preset small area and second image data representing the remaining portions of the
predetermined small area is detecfed by an intensity difference 'detecting component. Whether the
firs! picture data matches to a halftone image is determined by a halfione detection portion. When
the intensity difference is equal to or less than a first predetermined value, the first image data does
not correspond to the halftone image, and the intensity of the first image data is equal to or less than a second predetermined value, an intensity change pan changes the intensity bfthe first image
data to a predetermined low intensity.
The imemion discloscd in patent numbered US2009016ll69A1 an image processing device that reads image dam items from frames of an original image using an area sensor unil.
corrects the inclinations ofthe image data items using a correction unit; elc. a resolution setting
unit sets a resolution for displaying the original image, a necessary frame number acquisition unil
gathers the numberpf frames required to carry om the high resolution conversion based on the se,
a maximum frame‘number storage unit stores data of the maximum number of frames of the
acquired image data items, and a high-resolution qonversion unit acquires image data with a
resolution higher than the pixel sensor's regolution through interpolation.
The invention disclosed in patent numbered CN] 13284037B a ceramic watermark carrier
recovery system grounded on a deep neural network which comprises the way of originally,
detecting a ceramic three- dimensional image acquired in advance grounded on an image
segmentation network of Evo- BiSeNel to acquire a watermark carrier part; secondly, carrying out
watermark carrier planarization treatment grounded on a space metamorphosis network; and
eventually recovering the watermark carri'er grounded on GAN recovering the carrier and
perfecting the robustness of the watermark. The system adopts the regularized activation subcaste
to ameliorate the instantaneity of image segmentation, and utilizes a complication attention medium
to further optimize and prize the bitsy edge characteristics of the irregular image on the ceramic
face, so that the information of the watermark carrier is more effectively reserved; and recoverflng
the watermark carrier by using GAN, wherein the system comprises the robust training of the
watermark carrier, so that the robustness of the watermark is bettered.
Detailed description of the Invention:
Generally. the present disclosure is directed 10 systems and methods for hiding infonnali‘on
and images using deep learning In particular. the systems and methods oflhe prgsem disclosure
can leverage deep neural networks to more effectively hide information. for example. hiding a
secret image within a cover image, so that it may only be recovered by an intended recipient. By
using deep learning, such as deep neural networks. the sys|ems and methods of the present
disclosure can delermine the besl manner in which m encode information within a cover image 10
more effectively hide the image. In panicular. by using deep neural networks. a large amount of inibmmtion.‘ such as a full-size image, can be hidden within a cover image in such a way than the
hidden image can only be revealed by using a componem of the trained deep neural network
ensemble For example: an image preparation neural nelwork can receive as input a secret image mm is to be hiddcn. prepare the secrex image to be emzodcd wizhin 2: cover image and provide the
prepared secret image as oulpul. The prepared secret image and a cover image can then he provided
l0 an image hiding neura} nelwork as input and the image hiding neural network can creme a
walennarked image as output than includes lhe secret i‘mage hiddén within lhe cover image In
particular. the walennarked image can closely resemble 1116 cover image. For example. it may be
difficult to discem. at least visually. that the walemmrked image is in any way different from the
cover image. To recoi'er the secret image the watermarked image can be then provided as input to
2) simultaneously trained decoding neural network which provides as output a reconstruction ofthe
secret imagc
Preparation (nulqsats
For the training and Iesling ofthe walemuarking network. Cats and dogs dalasets were used as the
cover and secret images. respectively. The cover and secret image damsel coméined 10.000 and
1.0001raining samples and 8.000 and 600 tcsling samples‘ By using the data augmentation process.
a noisy watermarked dalasct (M') for the training of the extractor network. The testing samples
were not used in the training process to demonstrate the (__'cnera|ising_y and leaming capabilities of
the proposed scheme.
Training and Testing
The proposed walennarking technique was trained in two phases (i.e.. embedding network training
and exlrzxction network training). The mean squared error was used 10 compute ‘hé loss function
during the training of both nelworks For me training of the embedding network. the Adaptive
Momem Estimation optimizer was used because of its abilily lo continuously learn afler each
epoch, The iraining and validation oflhe embedding network where the loss (Ll) (Eq. I) during
each epoch is presented. The smaller gap between the training and \Ialidalién losses indicates that
the model cannot be categorized as overfilling. All the layers oflhe network applied the rectified
linear unil (ReLU) as the activation function except for the output layer. which used the sigmoidal
function [0 limit the range to (0.1). During the testing plmse._lhe PSNR and the SSIM were used to
evaluate the fidelity oflhe marked image.

The lasting PSXR was 44.48 dB and 1h? SSIM “:15 (L99. indicating ihe high fideliz) oi” lhc‘
watermarked images. Therefora embedded inlbrmalion was invisible 10 the human eye. The
exlmcmr network was trained using the noisy \x'alermarked images. which allo“ the network 10
learn. the noise fluctuations. This was done so that the exlraclor network could extract the sccrel
image even in cases where [he watermarked image contained some level ofnoise, The training and
validation oflhe extraction nelwork where the values oflhe loss (L2) (Eq, 2) during each Cpoch are
presented. The smaller gap between the training and validation loss indicales the mo-del learning
performance for the watennark extraction. The ADAM optimizer was used for each epoch for all
layers excepi for the output layer: which used the ReLU activation function and the sigmoidul
function.- During lhe testing phase‘ the. NC" (Nonnaliztd correlalion) value was determined 10
evaluate lhn‘ quality ofthe extracted walelmark image. On the test damsel. [he obtained NC score
was 090.
L2 = MSE(W. WI ) ---->Eq.2
Con wtlmimml Neural [Vanvork
Convolutional Neural Networks (CNNS) are used for tasks like image claésificalion. object
detection. and segmentation CNNS are employed in certain aspects ofthe walennarking process.
CNNS can be used to improve the robusxness and imperceptibility ofwaten’narks and also they help
to determine oplimal embedding locations within an image based on the visual features, thus
making lhe walennark less suspeptible to removal or alteration like cropping. and compression.
Furthennore. CNNs are used in assessing the impacl of walexmarking on image quality by
predicting distortions caused by the walemlark.

Embedding Network

Given [he cover (C) and secret (W) images. [he‘lalenl features of both C and'W are computed,
which are then concatenaled via the embedder network pc and pw. lnversely. the decoder network
(0W) Icarus a decoding function to dccodc the concatenated feature to obtain the marked image
(M). Here. the latenl representation of cover and mark images are denoted as C7. and W2.
respectivély. The encoder progressively decreases the size oflhe cover image feature map 10 make
il equal 10 the mark feature map so lhal the feature maps ofWZ and CZ can-be concatenated. Lalelz the decoder progressively increases the feature map [0 obtain the marked image.

Extractor Network
The extraction network is composed of a denoising(encoder-decoder network along with a
convolulinnal block. The extractor network extracts the embedded walermark image from the
watennarked image, Initially. a denoising aumencoder network is used to reduce noise effect (if
any) from the received data at receiver side. Later. the encoders are used 10 oblain the latent feature
from the denoised image and the cover imége. Here. the extracted Ialenl feature of‘the cover image
is subtracted from the marked lalcnl feature 10 obtain the residual of the marked image.
Subsequently. the obtained residual fealures are fancned Io l6.384 network paramaers. The CNN
block is used to make the flauened features dense and later concatenated and reshaped, Finally. the
decoder nelwork is used to '()b1.ain walexmark images by progressively increasing the rcshaped
feature map.
Performance A Italvxix
To evaluate the embeddingI and recovering performance of our scheme. we used three metrics to
measure the quality ote marked image and the recovered mark image. including the peak signallo-
noise ratio (PSNR). slrucliu‘al similarity index measure (SSIM) and normalised correlation (NC).
BRIEF DESCRIPTION OF DRA WING:
The figures illustrate exemplary embodiments oflhe invention
Figure l : Flow Chal'l indicating overall implementation of deep learning based digital image
walennarking using CNN algorithm. .
Figure 2 : Detailed architecture along with network configuration of the embedded network
Figure 3 : Detailed architecture along with network configuration of extraction network
Figure 4 : Overview of the Proposed model
Detailed description of the drawing:
The present invention generally relates to the model, which can be used to ensure copyright
protection through digital image watermarking using deep learning.
Figure ] depicts the flow chart indicating watermark insertion: extraction and removal of noise.

The cover image and secret image is given to the encoder network for generating a watermark
image. Before extracting a secret image from a watermarked image, noise can be removed from the
watermarked image using an auloencoder network .
Figure 2 depicts the step by step process of embedding a secfel image in the cover image. Given
the cover (C) and secret(W) images, the latent features of both C and W are computed, which are
then concatenated Via the embedder network pc and pw to generate watermarked images.
Figure 3 depicts the step by step process of extracting a secret image from a watermarked image.
The extractor network extracts the embedded secret image from the watermarked image. Initially,
a denoising autoencoder network is used to reduce noise effect. Finally. the decoder network is
used to obtain watemark images.
Figure 4 depicts the overall process of embedding and extracting 5 secret image. We use
convolutional neural networks (CNNs) for proposing a watermarking technique for digital images.
At first, latent features of cover and secret images are extracted using an encoder network and later
they are concatenated to generate a watermarked image‘ On the receiver side, a denouncing
auto encoder network is used for removing noise variations from the watermarked image and later
watermark is extracted from an image using a CNN.

We claim:
].The Deep CNN Image Watermarking technique ensures high invisibility and robustness,
and improves the performance significantly by up to 41% in robustness and 31% in
invisibility compared to other methods. This makes it a promising technique for traceable
copyright protection of digital images.
Deep CNN Image Watermarking: A technique that uses deep convolutional neural networks
to embed watennarks in digital images.
Invisibility: The abilify ofa watermark to be undetectable to the human eye.
Robustness: The ability ofa watermark to withstand attacks, such as cropping, resizing, and
compression.
'
Traceable‘ copyright protection: The ability to track the ownership ofa digital image.
2.Multi‘watermarking is a technique that embeds multiple watermarks into a single image,
each sewing a different purpose. This can be used to achieve a variety of goals, such as
copyright protection, tracking, and authentication. For example, one watermark could be used
to identify the owner of the'image, .while another could be used to track the image‘s
distribution Multi-watermarking can also be used to improve the robustness of the
walennarks against attacks.
3‘To improve the robustness ofa watermark against attacks, it can be embedded in multiple, -
domains, such as the frequency dofnain, the color domain, and the texture domain. This
makes it more difficult for attackers to remove or degrade the watermark, as they would need
to attack' the watermark in all of these domains. For example, the watermark can be embedded
'
in the frequency domain to make it resistant to cropping and resizing, and in the color domain
to make it invisible to the human eye. By embedding the watermark in multiple domains, it
can be made more robust against a wider range of attacks.

4. A mulIi-domain digital image watermarking system is a system that embeds 3 walennark
in both the frequency and spatial domains of an image using' deep neural networks. This
makes the watermark more robust against attacks: as it would be difficuh for an attacker to
remove or degrade the watermark in both domains. The system comprises a processor that is
configured to embed the watermark in the frequency and spatial domains ofthe image. The
processor uses deep neural networks to learn the optimal way_ to embed the watemark in
each domain. This' ensures that the watermark is embedded in a way that is both robust and
invisible.
5. Robust watermark extraction method comprising, applying advanced signal processing
techniques and machine learning algorithms to extract watermarks accurately from distorted
images, ensuring reliable watermark recovery even in the presence of severe image
alterations.
6.Contem—adaptive watermarking is a technique that embeds 3 watennark into an image in a
way tha‘ minimizes distortion and maximizes invisibility. by considering the image's
characteristics. This is done by analyzinglthe image‘s color, texture, and edge content, and
then adjusting the watermark embedding process accordingly. This makes content-adaptive
watermarking a more Irobust and effective watermarking technique .than traditional methods.
Color: The distribution ofcolors in the image.
Texture: The patterns of small variations in' color or intensity in the image.
Edge content: The presence of edges in the image.
By considering these image characteristics, the Walgreen embedding process can be
adjusted to minimize the distortion caused by the watermark, while still ensuring that the
watermark can be extracted later. This makes content—adaptive watermarking a valuable tool
for protecting digital images from copyright infringement and other malicious activities.