United States Patent Application: 0210192019

( 3 of 209 )

United States Patent Application	20210192019
Kind Code	A1
LWOWSKI; Jonathan Richard ; et al.	June 24, 2021

SYSTEM AND METHOD FOR DIGITAL STEGANOGRAPHY PURIFICATION

Abstract

Exemplary systems and methods are disclosed for removing steganography from digital data is disclosed. The method and system involve receiving a digital data. At least one processing device accesses a steganography purifier model. The at least one processing device includes at least a generator configured to scale a magnitude of individual data elements of the digital data from a first value range to a second value range. The scaled data elements are downsampled to remove steganography data embedded in the digital data and produce a purified version. The purified version is upsampled by interpolating new data elements between one or more adjacent data elements to provide an upsampled purified version. The magnitude of the data elements of the upsampled purified version are scaled from the second value range to the first value range to generate a purified output version.

Inventors:

LWOWSKI; Jonathan Richard; (San Antonio, TX) ; CORLEY; Isaac Alexander; (San Antonio, TX)

Applicant:

Name	City	State	Country	Type
Booz Allen Hamilton Inc.	McLean	VA	US

Assignee:

Booz Allen Hamilton Inc.
McLean
VA

Family ID:

76438456

Appl. No.:

17/123948

Filed:

December 16, 2020

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number
62949754	Dec 18, 2019

Current U.S. Class: 1/1

Current CPC Class: G06N 3/08 20130101; G06N 3/04 20130101; G06F 21/16 20130101; G06F 2221/0748 20130101; G10L 19/0212 20130101; G06T 1/005 20130101; G06T 3/4007 20130101; G06T 2201/0202 20130101; G10L 25/57 20130101

International Class: G06F 21/16 20060101 G06F021/16; G06T 3/40 20060101 G06T003/40; G10L 25/57 20060101 G10L025/57; G10L 19/02 20060101 G10L019/02

Claims

1. A system for removing steganography from a digital data, the system comprising: a receiving device configured to receive digital data including at least one of image data and audio data; and at least one processing device configured to access a steganography purifier model having at least a generator configured to: scale a magnitude of individual data elements of the digital data from a first value range to a second value range, downsample the scaled data elements to remove steganography data embedded in the digital data and produce a purified version of the digital data, upsample the purified version of the digital data by interpolating new data elements between one or more adjacent data elements to provide an upsampled purified version of the digital data, and scale the data elements of the upsampled purified version of the digital data from the second value range to the first value range to generate a purified output version of the received digital data.

2. The system according to claim 1, wherein the received digital data includes image data and the individual elements of the digital data are pixels.

3. The system according to claim 2, wherein if the image data includes video image data, the system comprising: a front-end device configured to parse the video image into a plurality of image frames and feed each image frame to the at least one processing device as the digital data.

4. The system according to claim 1, wherein the received digital data includes audio data.

5. The system according to claim 4, comprising: a front-end device configured to reshape a vector of the audio data into a matrix format and feed the matrix to the at least one processor as the digital data.

6. The system according to claim 5, wherein each individual element of the digital data is a sample of the audio data

7. The system according to claim 5, wherein the at least one processor is configured to generate the purified image as a matrix and reshape the matrix into a vector to generate purified audio data.

8. The system according to claim 1, wherein steganography purifier model of the at least one processing device includes a discriminator configured to distinguish between the purified image generated by the generator and a cover image of the digital data during a training mode.

9. A method for removing steganography from digital data, comprising: receiving, in a receiving device digital data including at least one of image data and audio data; scaling, in at least one processing device which having an encoder of a steganography purifier model, a magnitude of individual data elements of the digital data from a first value range to a second value range; downsampling, in the encoder architecture of the at least one processing device, the scaled data elements to remove steganography data embedded in the digital data and produce a purified version of the digital data; upsampling, in the at least one processing device having a decoder of a steganography purifier model, the purified version of the digital data by interpolating new data elements between one or more adjacent data elements to provide an upsampled purified version of the digital data; and scaling, in the decoder architecture of the at least one processing device, the data elements of the upsampled purified version of the digital data from the second value range to the first value range to generate a purified output version of the received digital data.

10. The method according to claim 9, wherein if the image data includes a video image data, the method comprises: parsing, in a front end device, the video image into a plurality of image frames; and feeding each image frame to the at least one processing device as a digital data.

11. The method according to claim 1, wherein if the received digital data includes audio data, the method comprises: reshaping, in a front-end device, a vector of the audio data to a matrix format, and feeding the matrix to the at least one processing device as the digital data.

12. The method according to claim 11, wherein the purified output version of the received digital data is generated as a matrix of values, the method comprising: reshaping the matrix into a vector to generate purified audio data.

13. The method according to claim 12, wherein the digital data is received in a data signal and formatted according to an application program interface.

14. A method of training a system for removing steganography from digital data, the system having a receiving device and at least one processing device accessing a steganography purifier model having at least a generator for generating purified digital data and a discriminator for distinguishing between the purified digital data and cover data, the method comprising: receiving, in the receiving device, a plurality of digital datasets including steganography, each digital datasets including one of image data and audio data; scaling, via the generator of the at least one processing device, a magnitude of individual data elements of the digital dataset from a first value range to a second value range; downsampling, via the generator of the at least one processing device, the digital dataset with the scaled elements to remove steganography data embedded in the digital data and producing a purified version of the digital dataset; upsampling, via the generator of the at least one processing device, the purified version of the digital dataset by interpolating data between one or more adjacent data elements to provide an upsampled purified version of the digital dataset; scaling, via the generator of the at least one processing device, the data elements of the upsampled purified version from the second value range to the first value range to generate a purified output version of the received digital dataset; receiving, in the discriminator of the at least one processing device, the purified output version of the received digital dataset and a reference digital dataset, which corresponds to the digital dataset received by the receiving device; and determining, via the discriminator of the at least one processing device, which of the purified version of the received digital dataset and the reference digital dataset contained steganography.

15. The method according to claim 14, comprising: feeding a determination result of the discriminator to the generator.

16. The method according to claim 14, comprising: adjusting one or more nodal weights of the generator based on whether the determination result of the discriminator is correct.

17. The method according to claim 15, comprising: wherein the nodal weights of the generator are adjusted so that a subsequent purified digital dataset has less steganographic content than a previous purified digital dataset.

18. The method according to claim 14, wherein if the image data includes video image data, the method comprises: parsing, in a front end device, the video image into a plurality of image frames; and feeding each image frame to the at least one processing device as a corresponding digital dataset.

19. The method according to claim 14, wherein if the received digital dataset includes audio data, the method comprises: reshaping, in a front-end device, a vector of the audio data to a matrix format, and feeding the matrix to the at least one processing device as a corresponding digital dataset.

20. The method according to claim 14, wherein the purified output version of the digital dataset is generated as a matrix of values, the method comprising: reshaping the matrix into a vector to generate a purified version of the audio data.

Description

FIELD

[0001] The present disclosure is related to methods and systems for removing steganography from digital data.

BACKGROUND

[0002] Steganography comprises various techniques of concealing data or information within other exposed or visible data or information used as a host. Digital image steganography involves having hidden text or data within a host image. Steganography increasingly poses a threat to the security of computer systems and networks. Bad actors can embed malicious payloads including program code, a shellcode, or scripts into an image, which is transmitted to a computer system or computing network. Once the image is downloaded to the computer system or network, the payload can be executed to control or exploit system or network operation.

[0003] Due to the highly undetectable nature of current state-of the art steganography algorithms, adversaries are able to evade defensive tools such as Intrusion Detection Systems (IDS) and/or Antivirus (AV) software which utilize heuristic and rule-based techniques for detection of malicious activity. However, these methods struggle to detect advanced steganography algorithms, which embed data using unique patterns based on the content of each individual file of interest. This results in high false positive rates when detecting steganography and poor performance when deployed, effectively making it unrealistic to perform preventative measures such as blocking particular images or traffic from being transmitted within the network. Furthermore, image steganalysis techniques are typically only capable of detecting a small subset of all possible images, limiting them to only detect images of a specific size, color space, or file format.

[0004] Known systems and techniques are designed to eliminate steganography contained in digital images using filtering to remove steganographic content making it unusable by a potential adversary. These systems and techniques, however, result in degradation of image quality that is perceptible to the human eye. See, for example, [5] "Attacks on steganographic systems" which uses generic filters to obfuscate/remove steganography; [6] "Pixelsteganalysis" which uses a neural network to build pixel distribution which is then used to manually remove steganographic content. This only works on lossy neural network based steganography which embeds images into other images; [10] "Anti-forensic approach to remove stego content from images and videos", which uses generic filters to obfuscate/remove steganography; [11] "On the removal of steganographic content from images" which uses generic filters to obfuscate/remove steganography; [12] "Optimal image steganography content destruction techniques" which uses generic filters to obfuscate/remove steganography; [13] "A novel active warden steganographic attack for next-generation steganography" which uses generic filters to obfuscate/remove steganography; [14] "Destroying steganography content in image files" which uses generic filters to obfuscate/remove steganography; [15] "A general attack method for steganography removal using pseudo-cfa re-interpolation" which uses generic filters to obfuscate/remove steganography; [16] "Active warden attack on steganography using prewitt filter" which uses generic filters to obfuscate/remove steganography; [17] "Denoising and the active warden" which uses generic filters to obfuscate/remove steganography. The entire content of each of the foregoing references is incorporated by reference.

SUMMARY

[0005] An exemplary system for removing steganography from digital data is disclosed, comprising: a receiving device configured to receive digital data including at least one of image data and audio data; and at least one processing device configured to access a steganography purifier model having at least a generator configured to: scale a magnitude of individual data elements of the digital data from a first value range to a second value range, downsample the scaled data elements to remove steganography data embedded in the digital data and produce a purified version of the digital data, upsample the purified version of the digital data by interpolating new data elements between one or more adjacent data elements to provide an upsampled purified version of the digital data, and scale the data elements of the upsampled purified version of the digital data from the second value range to the first value range to generate a purified output version of the received digital data.

[0006] An exemplary method for removing steganography from digital data is disclosed, the method comprising: receiving, in a receiving device, digital data including at least one of image data and audio data; scaling, in at least one processing device having an encoder of a steganography purifier model, a magnitude of individual data elements of the digital data from a first value range to a second value range; downsampling, in the encoder architecture of the at least one processing device, the scaled data elements to remove steganography data embedded in the digital data and produce a purified version of the digital data; upsampling, in the at least one processing device having a decoder of a steganography purifier model, the purified version of the digital data by interpolating new data elements between one or more adjacent data elements to provide an upsampled purified version of the digital data; and scaling, in the decoder architecture of the at least one processing device, the data elements of the upsampled purified version of the digital data from the second value range to the first value range to generate a purified output version of the received digital data.

[0007] An exemplary method of training a system for removing steganography from digital data is disclosed, the system having a receiving device and at least one processing device configured to execute or having access to a steganography purifier model having at least a generator for generating purified digital data and a discriminator for distinguishing between the purified digital data and cover data, the method comprising: receiving, in the receiving device, a plurality of digital datasets including steganography, each digital dataset including one of image data and audio data; scaling, via the generator of the at least one processing device, a magnitude of individual data elements of the digital dataset from a first value range to a second value range; downsampling, via the generator of the at least one processing device, the digital dataset with the scaled elements to remove steganography data embedded in the digital dataset and produce a purified version of the digital dataset; upsampling, via the generator of the at least one processing device, the purified version of the digital dataset by interpolating data between one or more adjacent data elements to provide an upsampled purified version of the digital dataset; scaling, via the generator of the at least one processing device, the data elements of the upsampled purified version from the second value range to the first value range to generate a purified output version of the received digital dataset; receiving, in the discriminator of the at least one processing device, the purified output version of the received digital dataset and a reference digital dataset, which corresponds to the digital data received by the receiving device; and determining, via the discriminator of the at least one processing device, which of the purified version of the received digital dataset and the reference digital dataset contained steganography.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Exemplary embodiments are best understood from the following detailed description when read in conjunction with the accompanying drawings. Included in the drawings are the following figures:

[0009] FIG. 1 illustrates a steganography purification system in accordance with an exemplary embodiment of the present disclosure.

[0010] FIG. 2 illustrates a steganography purification model of the processing device in accordance with an exemplary embodiment of the present disclosure.

[0011] FIGS. 3A-3E illustrate flow diagrams of a generator in accordance with an exemplary embodiment of the present disclosure.

[0012] FIGS. 4A and 4B illustrates a discriminator of the processing device in accordance with an exemplary embodiment of the present disclosure.

[0013] FIG. 5 illustrates a flow diagram of generating image data to train the generator and discriminator in accordance with an exemplary embodiment of the present disclosure.

[0014] FIG. 6 illustrates a method for removing steganography from digital data according to an exemplary embodiment of the present disclosure.

[0015] FIG. 7 illustrates a method for training a processor to execute a steganography purification model according to an exemplary embodiment of the present disclosure.

[0016] FIGS. 8A-8E illustrate examples of scrubbed images using various known steganography purification models in comparison with the steganography purification model of FIG. 2.

[0017] FIGS. 9A-9E illustrate examples of image differencing using various known steganography purification models in comparison with the steganography purification model of FIG. 2.

[0018] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description of exemplary embodiments is intended for illustration purposes only and is, therefore, not intended to necessarily limit the scope of the disclosure.

DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure are directed to methods and systems for steganography detection in a digital image. The one or more processing devices can be configured to access and/or execute a deep neural network for performing image steganography removal across a plurality of steganography algorithms and embedding rates, to remove steganography that is present in a digital image without degrading visual quality.

[0020] FIG. 1 illustrates a steganography purification system in accordance with an exemplary embodiment of the present disclosure.

[0021] As shown in FIG. 1, the steganography purifying system 100 can include a receiving device 102, a transmitting device 104, a front-end device 106, a processing device 108, an Input/Output (I/O) interface 110, a memory device 112, and a post-processing device 114. The receiving device 102 can include a combination of hardware and software components and be configured to receive digital from a remote computing device 120, such as a friendly or adversarial computing device. According to an exemplary embodiment, digital data or a digital dataset or data file can be received direct from the remote computing device 120. According to another exemplary embodiment, the digital data can be embedded and/or formatted in a digital signal associated with an application program interface (e.g., email), music platform, social media platform, etc. The receiving device 102 can be connected to the remote computing device 120 via a peer-to-peer connection 140 or through a network 150.

[0022] The peer-to-peer connection 140 can be configured for wireless communication without an intermediate device or access point. The network 150 can be configured for wired or wireless communication, which may include a local area network (LAN), a wide area network (WAN), a wireless network (e.g., Wi-Fi), a mobile communication network, a satellite network, the Internet, fiber optic cable, coaxial cable, infrared, radio frequency (RF), another suitable communication medium as desired, or any combination thereof.

[0023] The receiving device 102 can include a hardware component such as an antenna, a network interface (e.g., an Ethernet card), a communications port, a PCMCIA slot and card, or any other suitable component or device as desired for effecting communication with the remote computing device 120, a database 130 for storing digital data, and/or the network 150. The receiving device 102 can be encoded with software or program code for receiving digital data according to one or more communication protocols and data formats. For example, the digital data can include an image data (e.g., dataset, or data file) and/or an audio data (e.g., dataset, or data file). The receiving device 102 can be configured to process and/or format the received data signals and/or data packets which include digital data for steganalysis. For example, the receiving device 102 can be configured to identify parts of the received data via a header and parse the data signal and/or data packet into small frames (e.g., bits, bytes, words) or segments for further processing in the one or more processing devices 106. The receiving device 102 can be configured to feed the received digital data to the at least one processing device 106.

[0024] The receiving device 102 can include a front-end device 106 according to the type of digital data received. For example, the front-end device 106 can be configured to determine whether the digital data includes image data and/or audio data. The image data can be processed according to whether the images are still images (small images) or video images. The front-end device 106 can be configured to parse a received video image into a plurality of image frames and feed each image frame to the processing device 108 as digital data. According to an exemplary embodiment, the front-end device 106 can be configured to reshape a vector of audio data (e.g., dataset or data file) into a matrix format and feed the matrix to the at least one processor 108 as the digital data. For example, the front-end device 106 can be configured to parse the audio vector into a plurality of segments or pieces and recombine the pieces into a matrix format. It should be understood that the front-end device 106 can be integrated into the receiving device 102, the processing device 108, or be arranged as a wholly separate component, such as in a processor configured to execute or have access to software or program code to perform the above-described operations.

[0025] The processing device 108 is configured to generate a purified image based on the digital data received from the receiving device 102. According to exemplary embodiments of the present disclosure, the processing device 108 can include one or more hardware processors can be designed as a special purpose or a general purpose processing device configured to execute program code or software for performing the exemplary functions and/or features disclosed herein. The one or more hardware processors can comprise a microprocessor, central processing unit, microcomputer, programmable logic unit, or any other suitable processing devices as desired. The processing device 108 can be connected to a communications infrastructure 105 including a bus, message queue, network, or multi-core message-passing scheme, for communicating with other components of the computing device 100, such as the receiving device 102, the transmitting device 104, and an I/O interface 110.

[0026] As shown in FIG. 2, the processing device 108 can be configured to access a neural network architecture comprised of a steganography purifier model 200 having a generator 202 and a discriminator 204. The generator 202 can be configured as a neural network having a plurality of layers for processing individual elements of the digital data received from the receiving device 102. The individual elements of the digital data including pixels representing image data and samples representing audio data. The discriminator 204 can include a plurality of neural network layers configured to compare a purified image output from the generator 202 with a corresponding cover image. The discriminator 204 is activated only during a training mode, such that only the generator 202 is used during a mode for steganography purification.

[0027] FIGS. 3A-3E illustrate flow diagrams of a generator in accordance with an exemplary embodiment of the present disclosure. As shown in FIG. 3A, the generator 202 can include an encoder 300 and a decoder 302. As shown in FIG. 3B, the plurality of neural network layers encompassing the encoder 300 can include a normalization layer 304 configured to scale a magnitude of individual elements of the digital data from a first (e.g., original) value range to a second value range. For example, the original value range of individual pixel elements of image data (e.g., dataset or data file) can be scaled from a value of 0 to 255 to a value of 0 to 1. The scaled digital data can be fed to a two-dimensional (2D) convolutional layer 306 arranged with a rectified linear unit (ReLU) 308. The convolutional layer 306 includes a kernel of specified size and a plurality of filters can be configured to filter the scaled digital file and a rectified linear unit (ReLU) 308 can be configured to generate an output indicating the presence of a feature in the portion or area of the digital data analyzed through the kernel of the convolutional layer. According to an exemplary embodiment of the disclosure, the convolutional layer 306 can include a 9.times.9 kernel used with 64 filters. However, it should be apparent to one of skill in the art that a kernel of any specified size and any number filters can be used according to the desired results. The output of the ReLU 308 can be fed to a downsampling layer 310 configured to downsample the feature(s) of the digital data output by the convolutional layer 306 by a specified factor. The output of the downsampling layer 310 is fed to a residual block 318, a convolutional layer 336, and a batch normalization layer 338, which are discussed in further detail below. As shown in FIG. 3C, the downsampling layer 310 can include a plurality of sub-layers encompassing at least a pair of two-dimensional convolutional layers 312, 316 with a second ReLU layer 314 provided therebetween. According to an exemplary embodiment, the convolutional layer 312 can be configured with a stride of 2 to realize a downsampling factor of 2. The downsampling operation removes steganography data embedded in the digital data and produces a purified version of the digital data.

[0028] According to an exemplary embodiment, the front-end processing for audio data could be performed within the generator 202 rather than a front-end device 106, by replacing the 2-D convolutional layer 306 with a one-dimensional convolutional layer such that processing of the audio file could be performed without converting the audio data to a matrix format.

[0029] As shown in FIG. 3B, the output of the downsampling layer 310 can be fed to a residual block 318 having 16 residual blocks. FIG. 3D illustrates an exemplary residual block in accordance with an exemplary embodiment of the present disclosure. As shown in FIG. 3D, the residual block 318 can include an input layer 320, which feeds the output of the downsampling layer 310 to a convolutional layer 322 having a specified kernel size in combination with a specified number of filters. The output of the convolutional layer 322 is fed to a batch normalization layer 324 followed by a ReLU layer 326, a convolutional layer 328 of a specified kernel size in combination with a specified number of filters, and another batch normalization layer 330 to standardize the output of the convolutional layer 328. The output of the batch normalization layer 330 is summed 332 with an output from the input layer 320. The output of the summer 332 is fed to an output layer 334. The value at the output layer 332 of the residual block 318 is fed to a two-dimensional (2D) convolutional layer 336 followed by a batch normalization layer 338 to standardize the output of the 2D convolutional layer 336. The output of the batch normalization layer 338 is summed 339 with the original output of the downsampling layer 310 to form an encoder output 340 containing one or more purified features of the received (e.g., original) digital data. According to an exemplary embodiment, each convolutional layer 312, 316, 322, 328, 336 can have a kernel size of 3.times.3 in combination with 64 filters. It should be apparent to one of skill in the art that any specified kernel size and number of filters can be used to achieve the desired results.

[0030] FIG. 3E illustrates a plurality of neural network layers included in the decoder 302. As shown in FIG. 3E, the encoder output 340 is fed to an upsampling layer 342, which interpolates data between data elements of the encoder output 340. The upsampling layer 342 increases the shape of the encoder output 340 to the size of the original digital data received at the input of the encoder 300. The output of the upsampling layer 342 is fed to a two-dimensional convolutional layer 344 with a specified kernel size and specified number of filters followed by a ReLU activation layer 346. The output of the ReLU activation layer 346 is fed to another two-cushioning dimensional convolutional layer 348 having a specified kernel size and number of filters. The convolutional layers 344, 348 can have a kernel of any size in combination with a number of filters to achieve a desired result. In accordance with an exemplary embodiment, the convolutional layer 344 has a 3.times.3 kernel and 256 filters, and the convolutional layer 348 has a 9.times.9 kernel with 1 filter. The output of the second convolutional layer 348 is fed to a Tanh activation layer 350, 352 which is configured to denormalize the individual elements of the digital data back to the original value range. For example, the magnitude of each pixel value is rescaled to have a value which falls within the original value range (0 to 255) of the digital data received at the receiving device 102. The output of the Tanh activation layer 350, 352 is a purified version of the original digital data received at the receiving device 102.

[0031] The post-processing device 114 can be configured to perform any suitable processing on the purified digital data output from the decoder 302, to convert the purified digital data back into its original format. For example, if the original digital data received at the receiving device 102 includes the video images, the post-processing device can reformat and order the image frames output from the decoder 302 in order to reconstruct the original video data of the original digital data. In another example, the post-processing device 114 can be configured to reconstruct audio data included in the original digital data by reshaping the matrix of the purified digital data output from the decoder 302 into an audio vector. It should be understood that the post-processing device 114 can be integrated into the processing device 108 or be provided in a wholly separate component, such as in a processor configured to execute software or program code to perform the above-described operations.

[0032] FIGS. 4A and 4B illustrates a discriminator of the processing device in accordance with an exemplary embodiment of the present disclosure. The discriminator 204 can be activated in the processing device 108 based on whether the training mode of the steganography purification model is to be executed. In the training mode, the discriminator 204 compares the purified image output from the generator 202 to a cover image of the original steganographic image. According to an exemplary embodiment, the generator 202 and discriminator 204 can be accessed or executed in different processing devices 108 such that upon completion of the training process the specific processing device 108 configured to execute or having access to the generator 202 can be deployed in the specified application.

[0033] As shown in FIG. 4A, the input layer of the discriminator 204 is a normalization layer 400 which scales the input images from a value of 0 to 1 (0, 1) to a value in the range of -1 to 1. The data input to the normalization layer 400 includes both the purified image and the cover image as shown in FIG. 2. The output of the normalization layer 400 is fed to a convolutional layer 402 having a specified kernel size (e.g., 3.times.3) and a specified number (e.g., 512) of filters. The output of the convolutional layer 402 is fed to a ReLU layer 404. The output of the ReLU layer 404 is fed to a series of discriminator blocks 406 to 420. As shown in FIG. 4B, each discriminator block includes a convolutional layer 422 configured to have a specified kernel size (e.g., 3.times.3), a specified number of filters X, and a specified stride Y. The output of the convolutional layer 422 is fed to a batch normalization layer 424 followed by a ReLU layer 426. The series of discriminator blocks 406 to 420 are configured to reduce the number of model parameters. This result is achieved by decreasing the number of filters in the convolutional layer 422 for each subsequent discriminator block. For example, the discriminator blocks 406 to 420 can include an increasing number of 3.times.3 filter kernels, which increase by a factor of 2 from 64 to 512. Each of the discriminator blocks 406 to 420 has a specified stride value used to reduce the image resolution each time the number of features in a preceding convolutional layer is doubled. The output of discriminator block 420 is fed to a series of layers including a flattening layer 428, a 1024 dense layer 430, a ReLU layer 432, a 1 neuron dense layer 434, and a sigmoid layer 436. As a result of the last group of layers, the discriminator outputs a probability that the purified image matches the cover image.

[0034] See [19] "Deep residual learning for image recognition" which details the ResNet neural network architecture; [26] "Rectified linear units improve restricted Boltzmann machines" which details the ReLU activation function. The entire content of these is incorporated by reference.

[0035] During the training mode, steganographic images can be input to the encoder 300 of the generator 202. FIG. 5 illustrates a flow diagram of generating image data for training the generator and discriminator in accordance with an exemplary embodiment of the present disclosure. See [27] "Adam: A method for stochastic optimization" which describes an optimization algorithm for training the neural networks. The entire content of which is incorporated by reference.

[0036] As shown in FIG. 5, an image dataset of a plurality of digital images is used to create the reference image dataset and the other image datasets encoded with steganography according to one of a plurality of steganography algorithms. The image dataset can include a plurality of digital images being different in one or more of an image format, image content, image size, and color space. The digital images in the base image dataset also serve as reference (e.g., cover) images in that they do not include steganography. The image dataset can be generated by encoding base images with steganography according to one or more steganography algorithms. Each image is iteratively input into steganography analyzer SA1 to SA4, where each analyzer is configured to execute a specified steganography algorithm at a specified embedding rate. The steganography algorithms can include, for example, Highly Undetectable Steganography (HUGO), steganography utilizing a universal distortion function (S-UNIWARD), steganography based on a hill cipher technique (HILL), steganography based on wavelet obtained weights (WOW) or any other suitable steganography algorithm as desired. Each image of the base image dataset is iteratively input with data embedded at different embedding rates. According to an exemplary embodiment, the selected embedding rates can be any one or more embedding rates selected from 0% to 100%, and in an exemplary embodiment can include rates of 10%, 20%, 30%, 40%, and 50%, where 10% is the most difficult embedded data to detect. It should be understood that while FIG. 5 illustrates the use of five (5) steganography analyzers, the number of steganography analyzers is not limited to the ones shown or the number of possible steganography analyzers which can be used to train the system 100. See [20] "Break our steganographic system": the ins and outs of organizing BOSS, which details the BOSSBase dataset used to train the models described herein. See [25] http://dde.binghamton.edu/download/stego_algorithms/, which describes known implementations of steganographic algorithms (HILL, HUGO, WOW, S-UNIWARD). See [21] Using high-dimensional image models to perform highly undetectable steganography," which explains the HUGO steganography algorithm; [22] "A new cost function for spatial image steganography" which explains the S-UNIWARD steganography algorithm; [23] "Universal distortion function for steganography in an arbitrary domain" which explains the S-UNIWARD steganography algorithm; [24] "Designing steganographic distortion using directional filters," which explains the WOW steganography algorithm. The entire content of each reference is incorporated by reference.

[0037] The plurality of steganographic images are fed to the encoder 302 of the generator 202. The encoder generates a purified version of each steganographic image, by scaling down a magnitude of the individual elements of the image and filtering scaled elements to remove the steganography. As the image passes through the generator 202, weights of the purified image are fed to the decoder 304 which decodes the image by rescaling the magnitude of the individual elements to return the image back to its original size and interpolating new data elements between adjacent data elements of the rescaled image to form a purified image. The decoding includes an interpolation of individual elements of the image replace those elements which were removed by the encoder 302. The purified image is fed to the discriminator 204, which compares the purified image to a corresponding reference or cover image of the original steganographic image using a mean square error (MSE) loss function. According to an exemplary embodiment other loss functions, such as, mean absolute error (MAE) and root mean squared error (RMSE) can be used as applicable to achieve the desired results.

[0038] FIG. 6 illustrates a method for removing steganography from digital data according to an exemplary embodiment of the present disclosure.

[0039] As shown in FIG. 6, digital data encoded with steganography is input to the receiving device 102 (Step 600). The receiving device 102 determines whether the digital data includes image data or audio data such that further processing of the data by the front-end device may be necessary. For example, if the digital data includes video image data, the front-end device 106 parses the video data into a plurality of still frames. In another example, the digital data includes audio data. The audio data can be initially formatted as a vector of values. The front-end device 106 processes the audio data by reshaping the vector into a matrix of values. The processing device 108 receives the digital data from the front-end device into the generator 202. The encoder 300 of the generator 202 scales the magnitude of individual elements of the digital data from a first (e.g., original) value range to a second value range to reduce the size of the dataset or data file (Step 602). For example, for image data the magnitude of each pixel can be scaled from a value between 0 and 255 to a value between 0 and 1. The encoder 300 downsamples the scaled digital data to generate a purified version of the dataset or data file (Step 604). The purified version of the data is fed to the decoder 302 of the generator 202, where it is upsampled to resize the purified data to its original size observed at the input of the generator 202 (Step 606). The upsampling can include interpolating new data elements between the data elements of the enlarged dataset or data file to produce an upsampled purified version. The resized dataset or data file is then rescaled such that the magnitude of the individual elements in the upsampled purified version of the image data are converted from the second value range to the first (e.g., original) value range of the received digital data (Step 608). For example, the decoder scales the magnitude value of each individual element of the dataset or data file from a value between 0 and 1 to a value between 0 and 255. The rescaled image is output as the purified digital data. The decoder 302 feeds the purified digital data to the post-processing device 214. If the digital data input at the receiving device 102 included video or audio data, the post-processing device 214 reassembles the still video frames according to the original format and sequence of the received video data, and/or reshapes the matrix of the audio values into a vector according to the received audio data. See [34] "Voxceleb: a large-scale speaker identification dataset" which describes an audio dataset used to embed malware payloads using the LSB steganography algorithm; [35] "On the theory of filter amplifiers" which details the butterworth filter which used to filter the audio files after steganography removal; and [36] "Elements of statistical analysis" which details the hanning window used with the butterworth filter in [35]. The entire content of which is hereby incorporated by reference.

[0040] FIG. 7 illustrates a method for training a processor configured to execute or access a steganography purification model according to an exemplary embodiment of the present disclosure. The system configured with components or devices for executing the method has the receiving device 104 and the processing device 108 configured to access a steganography purifier model having at least a generator 202 for generating purified digital data and a discriminator 204 for distinguishing between the purified digital data and cover data during a training process. During the training process, the system performs the method of FIG. 6 for generating a purified image and performs the additional steps of distinguishing the purified image from the corresponding cover image as performed by the discriminator 204. See [4] Generative Adversarial Nets which describes a training framework used to optimize the neural network. The framework uses a 2.sup.nd neural network to "critique" the purifier neural network's output. See [18] Photo-realistic single image super-resolution using a generative adversarial network, that uses GANs to get optimal image quality (e.g., image super resolution). The entire content of these references is incorporated by reference.

[0041] The purified output image is fed to the discriminator along with a cover image corresponding to the related steganographic image (Step 700). The discriminator 204 attempts to correctly determine which input image is the cover image and which is the purified image (Step 702). The determination result of the discriminator 204 is fed to the generator 202 (Step 704). One or more nodal weights of the generator 202 are adjusted based on whether the determination result of the discriminator is correct (Step 706). The nodal weights of the generator 202 are adjusted so that subsequent purified digital data has less steganographic content than previous purified digital data. During the training process, the generator 202 and the discriminator 204 operate according to a generative adversarial network (GAN). For example, the generator 202 is configured to generate a purified image with the objective of having the discriminator 204 select the purified image as the cover image or increase the selection error rate of the discriminator 204. The generator 202 is trained based on whether it successfully deceives the discriminator 204. That is, as the success rate of deception is higher, a higher quality purified images is being generated. According to an exemplary embodiment, the GAN framework of the generator 202 and discriminator 204 can be trained for 5 epochs to fine tune the generator to produce more accurately purified images with high frequency detail of the original cover image. It should be understood, however, that any number of epochs can be used during the training mode as desired.

[0042] Turning back to FIG. 1, the transmitting device 104 can be configured to receive the purified digital data from the processing device 108 via the communication infrastructure 105. The transmitting device 104 can assemble the data into a data signal and/or data packets according to the specified communication protocol and data format for communication over the network 150 to another computing device. The transmitting device 104 can include any one or more of hardware and software components for generating and communicating the data signal via a direct wired or wireless link to a peripheral or remote computing device 120. The transmitting device 104 can be configured to transmit information according to one or more communication protocols and data formats as discussed above in connection with the receiving unit 102.

[0043] According to an exemplary embodiment, the I/O interface 110 can also be configured to receive the probability data from the processing device 108 via the communication infrastructure 105. The I/O interface 110 can be configured to convert the probability data into a format suitable for output on one or more output devices 130. According to an exemplary embodiment, the output device 130 can be implemented as a display device, printer, speaker, or any suitable output format as desired.

[0044] To compare the quality of the resulting purified images, the following metrics were calculated between the purified images and their corresponding steganographic counterpart images: Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [28], and Universal Quality Index (UQI) [29]. The MSE and PSNR metrics are point-wise measurements of error while the SSIM and UQI metrics were developed to specifically assess image quality. To provide a quantitative measurement of the model's distortion of the pixels to destroy steganographic content, a bit error ratio (BER) metric, which can be summarized as the number of bits in the image that have changed after purification, normalized by the total number of bits in the image.

[0045] The output results of the disclosed steganography purifier were baselined against several steganography removal or obfuscation techniques. The first method simply employs bicubic interpolation to downsize an original image of FIG. 8A by a scale factor of 2 and then resize the image back to its original size. As seen in FIG. 8B, the purified image using bicubic interpolation is blurry and does not perform well with respect to maintaining high perceptual image quality. The next baseline method consists of denoising filters using Daubechies 1 (db 1) wavelets [30] and BayesShrink thresholding [31].

[0046] An example of the resulting denoised image can be seen in FIG. 8C. It is notable that the wavelet denoising method is more visually capable of maintaining adequate image quality in comparison to the bicubic resizing method. The final baseline method compared to the disclosed steganography purifier (DDSP) uses the pretrained autoencoder prior to GAN fine tuning as the purifier. As seen in FIG. 8D, the autoencoder does an adequate job in maintaining image quality while purifying the image. Finally, the resulting purified image from the DDSP can be seen in FIG. 8E. The DDSP and the autoencoder's purified images have the best visual image quality, with the wavelet filtered image having a slightly lower image quality. Not only does the DDSP maintain very high perceptual image quality, it is quantitatively image purifier based on image quality metrics. As seen in Table I, the images purified using DDSP resulted in the greatest performance with respect to the BER, MSE, PSNR, SSIM and UQI metrics in comparison to all baselined methods. Since the DDSP model resulted in the highest BER at 82%, it changed the highest number of bits in the image, effectively obfuscating the most amount of steganographic content. Even though the DDSP model changed the highest number of bits within each image, it produced outputs with the highest quality as verified by the PSNR, SSIM, and UQI metrics, indicating that of the methods listed in Table I below, DDSP is the preferred method for steganography removal.

TABLE-US-00001 TABLE I Testing Results on the BossBase Dataset Model BER MSE PSNR SSIM UQI DDSP 0.82 5.27 40.91 0.99 0.99 AutoEncoder 0.78 5.97 40.37 0.98 0.99 Wavelet Filter 0.52 6942.51 9.72 0.19 0.50 Bicubic Inter. 0.53 6767.35 9.82 0.22 0.51

[0047] To provide additional analysis of the different image purification models, the original cover image can be subtracted from corresponding purified images allowing for the visualization of the effects caused by steganography and purification. As seen in FIG. 9A, when the cover image and the corresponding steganographic image are differenced, the resulting image contains a lot of noise. This is expected because the steganography algorithm injects payloads as high frequency noise into the images. The differenced bicubic interpolation purified image, seen in FIG. 9B, removes the majority of noise from the image. However, as discussed in the previous section, the bicubic interpolation method does not maintain good visual quality as it removes original content from the image. As seen in FIGS. 9C and 9D, both the denoising wavelet filter and autoencoder purifier do not remove the noise from the image. Instead, they both appear to inject so much additional noise into the image to obfuscate the steganographic content, that the image can be unusable. This is visually apparent in the noise located in the whitespace near the top building within the image. For both the wavelet filter and autoencoder, this noise is visually increased in comparison to the original steganographic image. Lastly, as seen in FIG. 9E, the DDSP model removes the noise from the image instead of injecting additional noise. This is again apparent in the whitespace near the top of the image. In the DDSP's purified image, almost all the noise has been removed from these areas, effectively learning to optimally remove the steganographic pattern, which we infer makes the DDSP have the highest image quality in comparison to other methods.

[0048] Transfer learning can be described as using and applying a model's knowledge gained while training on a certain task to a completely different task. To understand the generalization capability of our model, the DDSP model is tested against the purification of images embedded using an unseen steganography algorithm along with an unseen image format. Additionally, the DDSP can be tested against a purification method of audio files embedded with an unseen steganography algorithm.

[0049] To test the generalization of the DDSP model across unseen image steganography algorithms, the purification performance of the BOSSBase dataset in its original PGM file format embedded with steganographic payloads using LSB steganography [32] is recorded. The images can be embedded with malicious payloads generated using Metasploit's MSFvenom payload generator [33], to mimic the realism of an APT hiding malware using image steganography. Without retraining, the LSB steganography images were purified using the various methods. The DDSP model removed the greatest amount of steganography while maintaining the highest image quality. These results can be verified quantitatively by looking at Table II.

TABLE-US-00002 TABLE II Transfer Learning Results on the LSB Dataset Model BER MSE PSNR SSIM UQI DDSP 0.82 5.09 41.05 0.98 0.99 AutoEncoder 0.78 5.63 40.62 0.98 0.99 Wavelet Filter 0.53 6935.08 9.72 0.19 0.50 Bicubic Inter. 0.53 6763.73 9.83 0.22 0.51

[0050] The audio files were from the VoxCeleb1 dataset [34], which contains over 1000 utterances from over 12000 speakers, however we only utilized their testing dataset. The testing dataset contains 40 speakers, and 4874 utterances. In order to use the DDSP model without retraining for purifying the audio files, the audio files were reshaped from vectors into matrices and then fed into the DDSP model. The output matrices from the DDSP model were then reshaped back to the original vector format to recreate the audio file. After vectorization, a butterworth lowpass filter [35] and a hanning window filter [36] were applied to the audio file to remove the high frequency edge artifacts created when vectorizing the matrices. The models were baselined against a 1-D denoising wavelet filter as well as upsampling the temporal resolution of the audio signal using bicubic interpolation after downsampling by a scale factor of 2.

[0051] As seen in Table III, the pretrained autoencoder, denoising wavelet filter, and DDSP are all capable of successfully obfuscating the steganography within the audio files without sacrificing the quality of the audio, with respect to the BER, MSE, and PSNR metrics. However, the upsampling using bicubic interpolation method provides worse MSE and PSNR in comparison to the other techniques. This shows that those models are generalized and remove steganographic content in various file types and steganography algorithms. Although the wavelet denoising filter has slightly better metrics than the DDSP and the pretrained autoencoder, we believe that the DDSP model would greatly outperform wavelet filtering if trained to simultaneously remove image and audio steganography and appropriately handle 1-D signals as input.

TABLE-US-00003 TABLE III Transfer Learning Results on the VoxCeleb 1 Dataset Model BER MSE PSNR DDSP 0.67 650.12 37.28 AutoEncoder 0.67 650.14 37.28 Wavelet Filter 0.67 643.94 37.65 Bicubic Inter. 0.64 1157.01 35.54

[0052] The computer program code for performing the specialized functions described herein can be stored on a non-transitory computer-readable medium, such as the memory device 112, which may be memory semiconductors (e.g., DRAMs, etc.) or other non-transitory means for providing software to the processing device 108. The computer programs (e.g., computer control logic) or software may be stored in a memory device 132. The computer programs may also be received via a communications interface. Such computer programs, when executed, may enable the steganography purification device 100 to implement the present methods and exemplary embodiments discussed herein. Accordingly, such computer programs may represent controllers of the steganography purification device 100. Where the present disclosure is implemented using software, the software may be stored in a computer program product or non-transitory computer readable medium and loaded into the steganography purification device 100 using a removable storage drive, an interface, a hard disk drive, or communications interface, where applicable.

[0053] The processing device 108 can include one or more modules or engines configured to perform the functions of the exemplary embodiments described herein. Each of the modules or engines may be implemented using hardware and, in some instances, may also utilize software, such as corresponding to program code and/or programs stored in memory. In such instances, program code may be interpreted or compiled by the respective processors (e.g., by a compiling module or engine) prior to execution. For example, the program code may be source code written in a programming language that is translated into a lower level language, such as assembly language or machine code, for execution by the one or more processors and/or any additional hardware components. The process of compiling may include the use of lexical analysis, preprocessing, parsing, semantic analysis, syntax-directed translation, code generation, code optimization, and any other techniques that may be suitable for translation of program code into a lower level language suitable for controlling the processing device 108 to perform the functions disclosed herein. It will be apparent to persons having skill in the relevant art that such processes result in the processing device 108 being a specially configured computing device uniquely programmed to perform the functions described above.

[0054] It will be appreciated by those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims rather than the foregoing description and all changes that come within the meaning and range and equivalence thereof are intended to be embraced therein.

REFERENCES

[0055] Each of the following references is incorporated by reference in its entirety. [0056] [1] I. Cox, M. Miller, J. Bloom, J. Fridrich, and T. Kalker, Digital watermarking and steganography. Morgan kaufmann, 2007. [0057] [2] "Apt37." [Online]. Available: https://attack.mitre.org/groups/G0067/ [0058] [3] W. Bender, D. Gruhl, N. Morimoto, and A. Lu, "Techniques for data hiding," IBM systems journal, vol. 35, no. 3.4, pp. 313-336, 1996. [0059] [4] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," in Advances in neural information processing systems, 2014, pp. 2672-2680. [0060] [5] A. Westfeld and A. Pfitzmann, "Attacks on steganographic systems," in International workshop on information hiding. Springer, 1999, pp. 61-76. [0061] [6] D. Jung, H. Bae, H.-S. Choi, and S. Yoon, "Pixelsteganalysis: Destroying hidden information with a low degree of visual degradation," arXiv preprint arXiv:1902.11113, 2019. [0062] [7] T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, "Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications," arXiv preprint arXiv:1701.05517, 2017. [0063] [8] S. Baluja, "Hiding images in plain sight: Deep steganography," in Advances in Neural Information Processing Systems, 2017, pp. 2069-2079. [0064] [9] R. Zhang, S. Dong, and J. Liu, "Invisible steganography via generative adversarial networks," Multimedia Tools and Applications, vol. 78, no. 7, pp. 8559-8575, 2019. [0065] [10] P. Amritha, M. Sethumadhavan, R. Krishnan, and S. K. Pal, "Anti-forensic approach to remove stego content from images and videos," Journal of Cyber Security and Mobility, vol. 8, no. 3, pp. 295-320, 2019. [0066] [11] P. Amritha, M. Sethumadhavan, and R. Krishnan, "On the removal of steganographic content from images," Defence Science Journal, vol. 66, no. 6, pp. 574-581, 2016. [0067] [12] S. Y. Ameen and M. R. Al-Badrany, "Optimal image steganography content destruction techniques," in International Conference on Systems, Control, Signal Processing and Informatics, 2013, pp. 453-457. [0068] [13] A. Sharp, Q. Qi, Y. Yang, D. Peng, and H. Sharif, "A novel active warden steganographic attack for next-generation steganography," in 2013 9th International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2013, pp. 1138-1143. [0069] [14] F. Al-Naima, S. Y. Ameen, and A. F. Al-Saad, "Destroying steganography content in image files," in The 5th International Symposium on Communication Systems, Networks and DSP (CSNDSP'06), Greece, 2006. [0070] [15] P. L. Shrestha, M. Hempel, T. Ma, D. Peng, and H. Sharif, "A general attack method for steganography removal using pseudo-cfa re-interpolation," in 2011 International Conference for Internet Technology and Secured Transactions. IEEE, 2011, pp. 454-459. [0071] [16] P. Amritha, K. Induj a, and K. Raj eev, "Active warden attack on steganography using prewitt filter," in Proceedings of the International Conference on Soft Computing Systems. Springer, 2016, pp. 591-599. [0072] [17] C. B. Smith and S. S. Agaian, "Denoising and the active warden," in 2007 IEEE International Conference on Systems, Man and Cybernetics. IEEE, 2007, pp. 3317-3322. [0073] [18] C. Ledig, L. Theis, F. Husz'ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681-4690. [0074] [19] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. [0075] [20] P. Bas, T. Filler, and T. Pevn'y, ""break our steganographic system": the ins and outs of organizing boss," in International workshop on information hiding. Springer, 2011, pp. 59-70. [0076] [21] T. Pevn'y, T. Filler, and P. Bas, "Using high-dimensional image models to perform highly undetectable steganography," in International Workshop on Information Hiding. Springer, 2010, pp. 161-177. [0077] [22] B. Li, M. Wang, J. Huang, and X. Li, "A new cost function for spatial image steganography," in 2014 IEEE International Conference on Image Processing (ICIP). IEEE, 2014, pp. 4206-4210. [0078] [23] V. Holub, J. Fridrich, and T. Denemark, "Universal distortion function for steganography in an arbitrary domain," EURASIP Journal on Information Security, vol. 2014, no. 1, p. 1, 2014. [0079] [24] V. Holub and J. Fridrich, "Designing steganographic distortion using directional filters," in 2012 IEEE International workshop on information forensics and security (WIFS). IEEE, 2012, pp. 234-239. [0080] [25] J. Fridrich, "Steganographic algorithms." [Online]. Available: http://dde.binghamton.edu/download/stego algorithms/ [0081] [26] V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in Proceedings of the 27.sup.th international conference on machine learning (ICML-10), 2010, pp. 807-814. [0082] [27] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014. [0083] [28] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli et al., "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004. [0084] [29] Z. Wang and A. C. Bovik, "A universal image quality index," IEEE signal processing letters, vol. 9, no. 3, pp. 81-84, 2002. [0085] [30] I. Daubechies, Ten lectures on wavelets. Siam, 1992, vol. 61. [0086] [31] S. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image denoising and compression," IEEE transactions on image processing, vol. 9, no. 9, pp. 1532-1546, 2000. [0087] [32] P. Wayner, Disappearing cryptography: information hiding: steganography and watermarking. Morgan Kaufmann, 2009. [0088] [33] "Msfvenom." [Online]. Available: https://www.offensivesecurity.com/metasploit-unleashed/msfvenom/ [0089] [34]A. Nagrani, J. S. Chung, and A. Zisserman, "Voxceleb: a largescale speaker identification dataset," in INTERSPEECH, 2017. [0090] [35] S. Butterworth et al., "On the theory of filter amplifiers," Wireless Engineer, vol. 7, no. 6, pp. 536-541, 1930. [0091] [36] O. M. Essenwanger, "Elements of statistical analysis," 1986.

* * * * *

Current U.S. Class:	1/1
Current CPC Class:	G06N 3/08 20130101; G06N 3/04 20130101; G06F 21/16 20130101; G06F 2221/0748 20130101; G10L 19/0212 20130101; G06T 1/005 20130101; G06T 3/4007 20130101; G06T 2201/0202 20130101; G10L 25/57 20130101
International Class:	G06F 21/16 20060101 G06F021/16; G06T 3/40 20060101 G06T003/40; G10L 25/57 20060101 G10L025/57; G10L 19/02 20060101 G10L019/02