Week 10 Assignment Attached Files: 3339252.3340524.pdf 3339252.3340524.pdf – Alternative Formats (903.188 KB) Read the attached document. Please, bri
Week 10 Assignment Attached Files:
3339252.3340524.pdf 3339252.3340524.pdf – Alternative Formats (903.188 KB)
Read the attached document. Please, briefly describe the topic and include your views on the topic. Please, include any citations to support your views. Revisiting Data Hiding Techniques for Apple File System
Thomas Göbel, Jan Türr and Harald Baier
da/sec – Biometrics and Internet Security Research Group,
Hochschule Darmstadt, Germany
{thomas.goebel,harald.baier}@h-da.de
jan.tuerr@stud.h-da.de
ABSTRACT
Data hiding is an important part of anti-forensic research since
the continuous development of operating systems, file systems and
other software may close some previously known vulnerabilities
but will often inadvertently create new ones. Many of the currently
used file systems such as FAT, NTFS or ext4 have been thoroughly
analysed. There are quite a few theoretical approaches and also
some practical tools that help us to hide data in the existing file sys-
tems in different ways. For the Apple File System (APFS), the new
standard file system for all Apple devices, only part of the previous
work is transferable. There are only a few published forensic analy-
ses of APFS so far and some forensic tools like the Sleuthkit have at
least partially adapted APFS functionality. However, anti-forensic
techniques specific to APFS have not yet been explored.
This paper aims to introduce APFS and some of its noncritical ar-
eas which can be exploited to hide data. A recently published modu-
lar anti-forensics framework called fishy allows the implementation
of modules containing a file system interface and corresponding
data hiding techniques. After a short theoretical introduction to the
framework, we present, as a practical part of this work, specific data
hiding techniques for APFS which are implemented in a separate
module for fishy. Finally, the newly found techniques are evaluated,
e.g., on the basis of their detectability, stability and capacity.
CCS CONCEPTS
• Applied computing → Computer forensics; Evidence col-
lection, storage and analysis.
KEYWORDS
File system forensics, Anti-Forensics, Data hiding, Apple File sys-
tem, APFS, fishy
ACM Reference Format:
Thomas Göbel, Jan Türr and Harald Baier. 2019. Revisiting Data Hiding
Techniques for Apple File System. In Proceedings of the 14th International
Conference on Availability, Reliability and Security (ARES 2019) (ARES ’19),
August 26–29, 2019, Canterbury, United Kingdom. ACM, New York, NY, USA,
10 pages. https://doi.org/10.1145/3339252.3340524
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
ARES ’19, August 26–29, 2019, Canterbury, United Kingdom
© 2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-7164-3/19/08.. .$15.00
https://doi.org/10.1145/3339252.3340524
1 INTRODUCTION
The rise of connectivity and digitisation of personal and work spaces
brings with it an increased threat of cyber attacks. An important
part of understanding the growing threat is the early discovery of
vulnerabilities, attack methods as well as potential anti-forensic
techniques used to stop or slow down the forensic process. Of the
four elementary types of anti-forensics described by Ryan Harris [9],
data hiding is of special interest as it does not destroy or create any
additional data [9] and simply exploits already existing weaknesses.
There are many forms of data hiding like abusing file formats,
encryption1 or hiding data within other files, like splitting audio
and video tracks to hide coded messages [14]. According to the
anti-forensic taxonomy of Conlan et al. [2], our paper focuses on
file system manipulation. This particular part of data hiding uses
specific data structures and potential vulnerabilities of a file system
to hide data. While there are some hiding places that are present
in almost every file system, such as the use of file slack which is
described by Stephen P. Larson [11] as well as Knut Eckstein and
Marko Jahnke [4], most of them are specific to a particular file
system.
With the help of the recently released modular open-source anti-
forensic framework fishy2, multiple data hiding techniques can be
applied, whether they are unique to one file system or not. The
file systems implemented so far (FAT12, FAT16, FAT32, ext4 and
NTFS) consist of several data hiding techniques and an interface to
interpret the respective file system. A new module developed for
fishy adds both compatibility with some existing techniques and
new techniques specific to the pooled-storage file system APFS.
1.1 Motivation
The motivation for this paper and the newly developed fishy module
is based on the following two points:
• Although they play an important role in forensic research,
many tools that allow in-depth investigation of data hiding
methods are either undocumented, meaning that both the
source code and the original author is unknown, or they
focus on a single file system, making adjustments and modi-
fications almost impossible.
• Apple products play an important role in private households
today and are also gaining in importance for work environ-
ments. Because Apple systems are not as open and accessible
to third parties as most alternatives, the specific risk of a
1Some researchers do not consider encryption as a form of data hiding as it does not
hide the existence of the data and simply tries to limit access to the data [12].
2https://github.com/dasec/fishy, last visited 24.04.2019
https://doi.org/10.1145/3339252.3340524
https://doi.org/10.1145/3339252.3340524
https://github.com/dasec/fishy
ARES ’19, August 26–29, 2019, Canterbury, United Kingdom Göbel, Türr and Baier
targeted attack increases [1]. With the introduction of Ap-
ple’s new file system in mid-2017, which is used by both
MacOS and iOS operating systems, it is now possible to find
vulnerabilities before they are abused in malicious ways.
This paper therefore gives an overview of all methods found so
far to hide data within APFS. Additionally, it presents an open
and well documented module adding APFS capabilities including
the previously mentioned hiding techniques to the open source
framework fishy.
1.2 Contribution
This paper first gives a brief introduction to both the APFS file sys-
tem and the anti-forensic framework fishy. In order to understand
the hiding technique approaches it is important to first understand
the new APFS-specific data structures, which is why our description
covers the most important data structures and functions of APFS.
Furthermore, we give a brief explanation of the modular structure
of fishy and the structure of the newly developed APFS module
itself. An in-depth explanation of the framework can be found on
the GitHub address mentioned above or in the corresponding paper
by Göbel and Baier [6].
Since this paper focuses on the discovery and presentation of
potential APFS-specific hiding techniques, the second part of the
paper describes in detail the vulnerabilities of APFS that these
techniques exploit. We thereby present the exact functionality of
the five hiding techniques already implemented. Finally, we evaluate
the functionality of the new hiding techniques by taking a closer
look at the following three evaluation metrics: (i) Detectability, (ii)
Stability, and (iii) Capacity.
1.3 Outline
The rest of the paper is organised as follows: Section 2 outlines
known forensic analyses of APFS as well as forensic tools that
support the new file system. Section 3 gives a brief overview of
APFS as well as an introduction to its relevant data structures and
functions. Section 4 describes in detail the newly found hiding
techniques. This Section also gives a short description of fishy, the
framework used to implement these techniques. In Section 5, we
then evaluate the implemented hiding techniques and rate them
based on the evaluation metrics Detectability, Capacity and Stability.
Section 6 and Section 7 conclude the paper and discuss potential
future work.
2 RELATED WORK
The general idea of how this paper is approaching anti-forensic
research is based on knowledge gained by other researchers such as
Grugq [7] or Ryan Harris [9] who have already started to collect and
evaluate anti-forensic methods and principles for different systems
in 2002 and 2006 respectively.
To the best of our knowledge, no current research on APFS-
specific data hiding techniques has yet been published. However,
there are a few in-depth forensic analyses as well as a forensic
tool used to recover data from an APFS partition. The first of them
comes from Kurt Hansen and Fergus Toolan [8], who published
their analysis of APFS in September 2017. Their analysis is based on
the sparse information Apple published about APFS at this time3
as well as a manual examination of the file system.
Building on the aforementioned paper, Jonas Plum and Andreas
Dewald published a further analysis of the Apple file system [3].
Unlike their predecessors Hansen and Toolan, they generally use
the nomenclature defined by Apple.
An additional, independent forensic analysis was done in 2017 by
Laura Pfeiffer [15], achieving results similar to Hansen and Toolan.
Plum and Dewald added another paper to their previous work
describing different ways to recover data from an APFS image [17].
This paper focuses on the process of parsing the Apple file system,
which is needed to recover as many current and deleted files as
possible, and compared it to more universal approaches like file
carving. They also developed an open source tool called afro (APFS
File Recovery Options)4.
In September 2018 Apple released the first version5 of their
official APFS reference [10], confirming many of the previously
mentioned papers’ findings and also adding previously unknown
information.
The support and creation of forensic tools besides the already
mentioned afro tool has also grown regarding drivers like fuse6
and Paragon7, making APFS disk images mountable in Linux and
Windows environments, respectively. There are also some forensic
tools freely available yet, like the iBored8 hex editor that have at
least partial APFS support. The popular Sleuthkit framework falls
into the same category, having recently published a development
branch [18], which provides updated APFS-compatible versions of
some of its core features, such as the fsstat command.
In addition to the forensic tools, the already mentioned modular
anti-forensic framework called fishy exists, that enables implemen-
tation and testing of data hiding techniques on multiple file systems
[6]. For the purpose of this paper, the framework is expanded by
a module containing an APFS interface and the hiding techniques
mentioned in Section 4.
3 INTRODUCTION TO APPLE FILE SYSTEM
The structure of APFS is a mix of known ideas and functionalities.
Unlike its predecessor HFS+ it is not a journaling file system. To
ensure secure file system transactions, APFS employs both atomic
operations and a checkpoint system [8]. Additionally, APFS is a
double layered file system. The external layer, the Container, is a
managing entity, responsible for the entire file system. Volumes are
the internal layer and manage files and directories with the help
of B-trees [10]. While an implementation of APFS only has one
container, the amount of possible volumes depends on the size of
the container9. An interesting note is the role of the volumes. While
they act similar to traditional partitions in some ways, they are
completely different in others. The most glaring difference being
3https://developer.apple.com/library/archive/documentation/FileManagement/
Conceptual/APFS_Guide/Introduction/Introduction.html, last visited 29.04.2019.
4https://github.com/cugu/afro. Last visited 2019-04-05.
5There have been multiple updates as of April of 2019.
6https://github.com/sgan81/apfs-fuse. Last visited 16.04.2019.
7https://www.paragon-software.com/home/apfs-windows/. Last visited 16.04.2019.
8http://files.tempel.org/iBored/ by Thomas Tempelmann. Last visited 16.04.2019.
9The upper limit is 100 volumes.
https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html
https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html
https://github.com/cugu/afro
https://github.com/sgan81/apfs-fuse
https://www.paragon-software.com/home/apfs-windows/
http://files.tempel.org/iBored/
Revisiting Data Hiding Techniques for Apple File System ARES ’19, August 26–29, 2019, Canterbury, United Kingdom
Container
Superblock
Container
Object
Map
Volume
Superblock
Checkpoint
Areas
Container
Superblock
Metadata
Container
State
Metadata
Additional
Volumes
Volume
Object
Map
Extent
and
Root
trees
Figure 1: Simplified structure of an APFS container, adapted
from Plum [17].
Node
Information
Entry
Headers
Potential
empty
space
Entry
Value
Figure 2: Simplified structure of an APFS node.
that, barring potential reserved blocks or quotas, they have a dy-
namic size they share with all other volumes as blocks are allocated
on a container level [3]. This feature is called Space Sharing.
An interesting challenge to overcome for data hiding purposes
is the way APFS references different structures. APFS distinguishes
between three different object types: (i) Ephemeral, (ii) Virtual, and
(iii) Physical. While virtual objects are always stored on disk and
ephemeral objects are generally stored in memory and only written
to disk as part of a checkpoint [10], they are referenced in the
same way. Their object ID, which is stored in a universally used
object header, is usually determined by the container superblock
field nx_next_oid. B-trees are used to map these object IDs to block
addresses. Physical objects are always written to disk and their
object IDs are their block addresses.
Figure 1 is a simple representation of the structure of an APFS
container. The data structures relevant to the hiding techniques
shown in Section 4 are the container and volume superblocks as
well as their respective object maps and the nodes contained in the
volume B-trees. Furthermore, the checkpoint areas, specifically the
Container Superblock Metadata, are of interest as well.
Another important fact of APFS is that all file system structures,
with the exception of the allocation bitmap, are stored as objects
and given object headers. These 32 byte sized headers contain
a checksum among other information. The checksum is a 64-bit
version of Fletcher’s checksum with the input being the entire block
without the checksum field [3].
There are two different versions of superblocks, one for each
layer. A copy of the current version of the container superblock is
usually found at block 0 of the container. It contains much of the
overarching information about the container, such as references to
its object map, a list of volumes and elementary information like
the system’s block size. The volume superblock acts as a similar
managing instance for its respective volume, containing informa-
tion about the size, name and contents of the volume as well as
references to extent trees and the volume object map.
Nodes are some of the most important data structures in APFS,
as they contain most of the information needed to interpret the
file system. The node structure can be seen in Figure 2. In addition
to general information about the node, such as its position in a
B-tree, it contains a table of contents that specifies the location of
the nodes’ entries [10]. A node’s entries vary drastically depending
on the node’s application within the file system. The entries are
split in key and value pairs, with the key determining the entry type.
The key and value areas are usually separated by free space, so new
entry keys and values can be added. Among the most important
entry types for this paper is type 3, which represents the inode and
whose structure can be seen in Table 1.
Checkpoints are a core feature of APFS. They represent past
states of the container by saving important structures and data. The
container superblock, memory states at the time of the checkpoint
and allocation structures, such as the space manager, are stored
in the Checkpoint Areas depicted in Figure 1. Object maps and
volume superblocks are stored in different places in the container
and referenced by the corresponding structures. This function has
positive impacts upon data hiding, like potentially bigger capacities
for techniques that use these structures. There also is a negative
impact since some crucial information about the processes the
checkpoint system uses are still unknown. It is not known how
many checkpoints a container can have10 or when exactly a new
checkpoint is created. More importantly, it is not known how old
structures are overwritten. If entire blocks are erased before a new
checkpoint is created, techniques, that use the slack space of these
structures, suffer from significant stability deficits.
4 HIDING TECHNIQUES
All presented hiding techniques, unless otherwise mentioned, are
developed and integrated into the anti-forensic data hiding frame-
work fishy, which therefore gets an additional APFS interface.
fishy is a python-based modular framework developed to be an
open solution for data hiding research. So far, four major file system
modules (NTFS, FAT, ext4, APFS) are available in the framework.
10Initial research using images of different sizes indicate that it is tied to the size of
the container.
ARES ’19, August 26–29, 2019, Canterbury, United Kingdom Göbel, Türr and Baier
The modules contain specific hiding techniques as well as file sys-
tem interfaces. These interfaces use self-programmed parsers or
other open source tools such as Construct11 to interpret file sys-
tem images. The framework already provides multiple wrappers
for its hiding techniques as well as a command line interface for
interaction with the user. In particular, all techniques are not only
able to write data, they can also read and erase12 the data that was
previously written into the file system. In order to find the hidden
data later on, every write method provides metadata relevant to the
technique, such as the size or offset of the hidden data, which is then
turned into an encrypted JSON file. If needed, the techniques also
calculate the structure’s checksum after data is written or erased.
To calculate the checksum after modifying an APFS structure the
algorithm developed by Jonas Plum [16] is used.
Most of the hiding techniques presented in the following Section
use free, reserved or otherwise unimportant space found in the file
systems’ inodes. These areas are highlighted by the bold text in
Table 1. The only exception is the Superblock Slack technique which
uses both types of superblocks and their respective object maps.
Table 1: Contents of an inode. Bold fields indicate the use of
a particular field for one of the hiding techniques.
Position Size Name
0 8 parent_id
8 8 private_id
16 8 create_time
24 8 mod_time
32 8 change_time
40 8 access_time
48 8 internal_flags
56 4 nchildren_or_nlink
60 4 def_protection_class
64 4 write_gen_counter
68 4 bsd_flags
72 4 owner
76 4 group
80 2 mode
82 2 pad1
84 8 pad2
92 variable extended_fields
4.1 Superblock Slack
Write to
Container
Superblock
Slack
Write to
Container
Object
Map
Slack
Write to
Volume
Superblock
Slack
Write
to
Volume
Object
Map
Slack
Load
next
list
entries
Figure 3: Visual representation of superblock slack target
choice.
Although this technique is not only usable in APFS13, the im-
plementation for APFS is somewhat unique due to its usage of two
11https://construct.readthedocs.io/en/latest/, Last visited 24.04.2019.
12The hidden data is overwritten with zeroes.
13For example, the ext4 module implemented in fishy already has a technique exploiting
the slack created by superblock data structures [6].
Superblock
Slack Main
Method
Data
Stream
Superblock
Slack
Submethods
Offset of
structure
Data
Data
Structure
Hide Data at
Offset
Return size of
hidden data
Calculate
checksum
Figure 4: Visual representation of superblock slack write al-
gorithm.
different superblock types and the adjacent object maps as well as
the effects of the checkpoint feature discussed in Section 3.
The block addresses of all relevant structures need to be collected
before any data can be hidden. For that purpose an external class
called Checkpoints gathers all four used structures in separate lists.
This happens in four steps. First, the container superblock is added
to the respective list. Second, the address of the attached object map
is extracted and stored in a separate list. Third, using the object map,
all current volume superblocks are found and added to a another
list. Finally, the object maps attached to each volume superblock
are stored in a fourth list. To expand the capacity of this technique,
older versions of these structures are used as well by following the
previously mentioned steps after locating all previous container su-
perblocks in the Checkpoint Areas seen in Figure 1. The information
to locate the Checkpoint Areas is found in a dedicated section in the
container superblock. All lists are sorted by the objects’ transaction
IDs, with the newest object located at the beginning of each list.
The hiding technique itself requests all four lists before starting
the hiding process. The write method itself calls four submethods,
one for each used structure type. The main method transmits the
corresponding address as well as the stream containing the data it
is supposed to hide to each submethod which in turn writes to the
respective structure’s slack space and returns the size of the written
data. After successful completion of a submethod, the checksum
for the used structure is recalculated. This process is visualised in
Figure 3 and Figure 4.
To recover or erase the hidden data, the method creates a meta-
data file containing four lists, one for each structure type filled
with the offsets of all used structures of that type as well as a field
containing the full size of the hidden data.
Due to the unknown nature of the checkpoint write system14
during development, a method that only uses older versions of the
structures when a higher capacity is needed was chosen. As seen in
Figure 3, the technique writes into the first entry of each list before
choosing an older and potentially less stable hiding space.
4.2 Write-Gen-Counter
Besides the nodes’ version numbers in the universal object headers,
every inode entry in these nodes has its own versioning method.
14It is unknown when exactly new checkpoints are created and how old checkpoints
are overwritten.
https://construct.readthedocs.io/en/latest/
Revisiting Data Hiding Techniques for Apple File System ARES ’19, August 26–29, 2019, Canterbury, United Kingdom
Data
Stream
WriteGen
Counter
Main
Method
Data
WriteGen
Counter
Submethods
Chunk of Data
& Inode offset
Inode
Hide chunk at
calculated offset
Calculate
Checksum
Figure 5: Visual representation of Write-Gen-Counter write
algorithm.
The inode versioning is being managed by write_generation_counter,
a 4 byte counter that is increased whenever the inode or its data is
modified. When reaching its limit the counter restarts from 0 [10],
making it a possibly undetectable hiding space since there are no
disallowed values within the fields 4 byte range.
Similar to the Superblock Slack technique described before, the
first step this and all subsequent techniques must perform is parsing
the target image and extracting the addresses of all nodes containing
inode entries as well as the exact offset of each inode entry. This is
done by an auxiliary class called InodeTable. This class gathers all
volume object maps and parses the referenced B-tree and its nodes
for inode entries. Before returning a tuple list containing the node
and inode addresses, the root nodes are removed since they only
contain references to inodes.
The process of hiding the data itself is quite simple and can be
seen in Figure 5. First, the stream of data that is supposed to be
hidden is split into chunks of 4 byte. An inode address is extracted
from the list received from InodeTable and given to a submethod
alongside one of the 4 byte chunks. The submethod then calculates
the exact offset of the hiding space and writes the contents of the
chunk to this address. After every written chunk, the respective
node’s checksum is newly calculated.
The metadata saved to access the hidden data at a later time
consists of lists of the used inode addresses and node addresses.
4.3 Timestamp Hiding
Similar to other modern file systems (e.g., btrfs, ext4, NTFS), APFS
offers 64-bit nanosecond timestamps. Exemplary techniques ex-
ploiting the nanoseconds part of timestamp fields already exist for
ext4 [5] and NTFS [13], which are used as a basis for the APFS
implementation. Although the currently implemented version of
this technique only uses the highlighted timestamps in Table 1, it
should be mentioned that APFS also has other structures, such as
volume superblocks, which contain usable timestamps.
Again, the address is extracted from the list returned by the In-
odeTable class. This address, the data stream and a number ranging
from 0 to 3 is given to a submethod that calculates the exact offset
and writes data to the first 4 bytes of the timestamp. The number
transmitted is an indicator of what timestamp the data is written to.
If the number exceeds 3, it is set to 0 and a new inode is targeted.
This means that the current implementation of this technique writes
to all four timestamps present in an inode. By default, our imple-
mentation not only ensures that all timestamps are written to, but
Timestamp
Hiding
Main
Method
Offset
of
Inode Chosen
timestamps
of Inode
used?
Yes: Skip to next
Inode
No: Offset
given to
Submethods
Data
Stream
Timestamp
Hiding
Submethods
Data
Return size of hidden
data, increase
timestamps used
counter
Calculate
Checksum
Hide data
in
chosen
timestamps
Inode
Figure 6: Visual representation of timestamp hiding write
algorithm.
also makes it possible to write to a specific timestamp with minimal
change. The complete hiding process is visualised in Figure 6.
In order to recover hidden data, the metadata file of this tech-
nique contains used node and inode addresses as well as the size of
the hidden data.
4.4 Extended Field Padding
Space occupied by extended fields of
an inode
Extended
Field 1,
Size = 17
Offset
0
Offset
17
Offset
24
Padding,
Size = 7
Extended
Field 2,
Size = 20
Offset
44
Offset
48
Padding,
Size = 4
Figure 7: Visual representation of extended field padding
generation.
The inode and directory record entry types have a unique feature
called Extended Fields that follow the regular entry contents. Usually,
these fields’ sizes are a multiple of 8 bytes. However, some of these
fields have variable sizes based on their content. If a field is not the
right size, it is enlarged to the next possible multiple of 8 bytes by
adding a padding field as seen in Figure 7. The programmed version
of this technique only writes to extended fields found in inodes
since extended fields seem to be more consistent there. Directory
records rarely use this feature in all tested images.
Like the previous hiding techniques, this method also extracts
the inode addresses from InodeTable. The difference here is that the
addresses of the hiding spaces are not always at a fixed offset inside
the inode. Instead, there are three additional steps. First, the table
of contents for the extended field of an inode, as seen in Table 2,
has to be interpreted and the number of extended fields has to
be extracted. Second, the sizes of the extended fields have to be
extracted from the list of extended field headers as seen in Table 3
ARES ’19, August 26–29, 2019, Canterbury, United Kingdom Göbel, Türr and Baier
following xf_blob15. Third, the sizes have to be used to determine if
there is any padding among this set of extended fields and if so, the
size of the padding has to be calculated16. Once all paddings are
calculated, the data stream and a tuple list containing the padding
addresses, the padding sizes as well as the inode addresses are given
to a submethod. The submethod writes to the different padding
locations and returns a similar list with used addresses, the size
of the padding at each used address and the corresponding node
addresses. In a final step, the correct checksums are calculated.
In this case, the created metadata contains lists of padding offsets,
padding sizes and node addresses.
Table 2: The table of contents for extended fields.
Position Size Name
0 2 number_of_xfields
2 2 size_of_xfields
4 number_of_xfields*4 xfield headers
Table 3: The structure of extended field headers.
Position Size Name
0 1 xfield_type
1 1 xfield_flags
2 2 xfield_size
4.5 Inode Padding
Each inode entry of a node has two padding fields of 2 and 8 bytes
each, which separate the entry value contents from the dynamic
extended fields17. This …
