Archive

Posts Tagged ‘ntfs file system’

NTFS File System Frequently Asked Questions Part I

November 19th, 2009 Comments off

Q: Is the boot limitation for NTFS still 7.87GB?

A: No. NTFS volume size limit is 2TB.

Q: Where can I get a lot of details about the NTFS encryption and security ?

A: Security (NTFS) and Encryption (NTFS,NTFS5) are wide topics.

You can get a lot of information about it on Microsoft’s MSDN Web Site (http://msdn.microsoft.com)

Security Topics: File Security and Access Rights

Encryption Topics: File Encryption , Encryption, Sparseness, and Reparse Points

Q: How do you lock files from other machine users on NTFS file system ?

A: NTFS has built-in security feature. Owner of the object can assign certain rights to certain users to restrict access to the object.

If file or folder is located on NTFS, just go to its Properties in Windows Explorer and on the Security tab add users being able access the object, and then remove Everyone user from users list, or restrict its rights.

Q: For Windows 2000 Professional using NTFS — Must All partitions be NTFS or can a FAT32 partition be functional as well?

A: Windows 2000 Professional as long as other Windows 2000 and XP family operating systems have full support for FAT32 and FAT16 file systems, as long as CDFS, HPFS, etc.

Operating system communicates with file system via logical level Win32 API that, in turn, redirects function calls to physical level (file system drivers), so if proper file system driver is installed, operating system can access and work with file system properly.

Drivers for NTFS, NTFS5, FAT12, FAT16, FAT32, CDFS are always installed when you install Windows 2000 / XP.
Drivers for NTFS, FAT12, FAT16, CDFS are always installed when you install Windows NT, there is no built-in driver for FAT32 in Windows NT.

Q: How do you format a blank hard disk drive to NTFS or NTFS5 ?

A: There are no standard utilities to format HDD to NTFS from DOS.

However there are solutions:

  • Attach HDD to another machine having Windows NT installed if you want to format to NTFS, or to Windows 2000 / XP if you want to format it to NTFS5. Then format drive using Disk Manager utility that is included in OS.
  • You can start Windows NT / 2000 / XP installation using bootable CD-ROM. On first steps of installation you will be asked about target location and you will be suggested to format the partition to NTFS. Go this way, and after format is completed, just cancel the installation process.

Q: Is there any problem with my games if I installed it under NTFS filesystem?

A: NTFS file system itself cannot cause problems to any software including games, because NTFS is just a way of data storage.

Software in Windows accesses files via upper-level Win32 API. Win32 API, in turn, redirects function calls to the drivers for the particular file system (NTFS/FAT/CDFS, etc.). Thus software, generally speaking, is not aware of file system it is installed on.

What could cause problems with games is Operating System itself. As long as Windows NT / 2000 / XP are more secure operating systems, and do not support 16-bit device drivers and real mode, some of games that work well under MS-DOS, Windows 95 / 98/ ME might not work under Windows NT / 2000 / XP.

Q: When Data is written to the NTFS disk at what position does in start the write? EG Assuming a file has been deleted (and deleted from the recycle bin if applicable) would it first overwrite this file, or would it start at the next totally unused cluster?

A: It depends on many factors including file system fragmentation, free space, etc.

In most cases it would start with a next totally unused cluster.

Q: When deleting a file at work from a HDD what is the best way to ensure that the file cannot be recovered? ie that the data no longer exists on the drive, rather than just resetting the flag of the relevant file/cluster? Is there any way NTFS will do this? Registry hacks e.t.c?

A: There are no standard mechanisms for this.

Please use third party privacy software (such as Disk Wiper feature in ZDelete) to eliminate unused MFT entries and overwrite clusters containing deleted data.

NTFS File System Frequently Asked Questions Part II

November 19th, 2009 Comments off
Q: Is it possible to convert a FAT32 Hard Drive to NTFS without losing all data on the drive? I like to change from FAT32 to NTFS, my operating system is Windows XP PRO, how can I do that? Without the lost of my programs?

A: Standard Windows utility that is called CONVERT serves this purpose

Just go to the Command Prompt and execute the command:

	C:\> CONVERT  C:  /fs:ntfs

Where C: is a name of the drive you want to convert.

After machine re-boot conversion process will start and you’ll have your FAT32 converted to NTFS without of data loss.

Q: How does NTFS compared to FAT32 in Windows XP, and which is faster?

A: NTFS has much more built-in features than FAT, so generally it is a bit slower.

However it depends on many factors such as cluster size, average file size, etc.

For example, NTFS can keep small files inside MFT entry, so if the file size is less than cluster size, most likely it will be accessed much faster on NTFS than on FAT.

Generally speaking the performance of NTFS on large volumes is higher than performance of FAT32. NTFS performance on small volumes is lower than performance of FAT/FAT32.

Q: How can I copy files from a hard drive formatted to NTFS, to a FAT32 hard drive ?

A: You probably asking about Windows NT that does not support FAT32.

There are third party FAT32 drivers for NTFS, or you can use FREE NTFS Reader to copy files in DOS environment. Just make sure that your DOS supports FAT32. You can use Bootable Floppy Creator to prepare such a floppy containing DOS and NTFS Reader for DOS.

Q: Which version of NTFS is installed on my Windows XP system ?

A: The following versions are currently available:

  • NTFS v1.2 on Windows NT
  • NTFS v3.0 on Windows 2000
  • NTFS v3.1 on Windows XP
Q: When I use the following command “FORMAT” on a volume (Windows XP) what is really written on this volume ?

A: Clean Master File Table (MFT) containing some system records is created for the volume.

Q: I am using a 249 megabyte drive as a backup drive on my xp system. I have it formatted in NTFS and compressed, yet the size of the drive is still the same as before I compressed it. Why?

A: Actual disk size cannot be changed. By applying compressed attribute for the volume you just ordered operating system to try to compress any object that will be placed there.

If object that is placed onto the volume can be compressed, operating system compresses it and it takes less space on the drive than uncompressed one. Thus more free space is left on the drive for other data.

Q: The files I place on the compressed drive are only compressed from 1.15MB to 1.14 MB , is it normal this should be only 100kb of compression per MB?

A: Compression on NTFS uses modified LZ77 algorithm. It is very fast but not always effective.

If works pretty well for the files/documents containing a number of repeating sequences of symbols. Example of such files types: text files, RTF, BMP, HTML files, etc…

For other file types, such as binaries, GIF, JPG, ZIP files, etc. this compression algorithm is not useful so that these files might not be compressed at all.

Q: Could I read file from my pc running windows XP with NTFS5 file system, from a machine under windows 95 on the same network?

A: Surely you can do it, if you configure Networking properly, i.e. create Network Share on WinXP for the folder where file is located and assign proper access rights to the share.

After performing these procedures if you can lookup WinXP machine across the Network you’ll be able to see this network share from Windows 95 and access files inside.

Q: Which is better? NTFS or NTFS5?

A: As for advances in technologies the latest versions are usually better than previous ones.

In addition to all NTFS features, NTFS5 has support for Encryption, Disk Quotas, Sparse Files, Reparse Points, Volume Mount Points.

How NTFS File System Works: NTFS Physical Structure (6)

September 17th, 2009 Comments off

NTFS Physical Structure Last Access Time

Each file and folder on an NTFS volume contains an attribute called Last Access Time. This attribute shows when the file or folder was last accessed, such as when a user performs a folder listing, adds files to a folder, reads a file, or makes changes to a file. The most up-to-date Last Access Time is always stored in memory and is eventually written to disk within two places:

  • The file’s attribute, which is part of its MFT record.
  • A directory entry for the file. The directory entry is stored in the folder that contains the file. Files with multiple hard links have multiple directory entries.

The Last Access Time on disk is not always current because NTFS looks for a one-hour interval before forcing the Last Access Time updates to disk. NTFS also delays writing the Last Access Time to disk when users or programs perform read-only operations on a file or folder, such as listing the folder’s contents or reading (but not changing) a file in the folder. If the Last Access Time is kept current on disk for read operations, all read operations become write operations, which impacts NTFS performance.

Note: File-based queries of Last Access Time are accurate even if all on-disk values are not current. NTFS returns the correct value on queries because the accurate value is stored in memory.

NTFS eventually writes the in-memory Last Access Time to disk as follows.

Within the file’s attribute

NTFS typically updates a file’s attribute on disk if the current Last Access Time in memory differs by more than an hour from the Last Access Time stored on disk, or when all in-memory references to that file are gone, whichever is more recent. For example, if a file’s current Last Access Time is 1:00 P.M., and you read the file at 1:30 P.M., NTFS does not update the Last Access Time. If you read the file again at 2:00 P.M., NTFS updates the Last Access Time in the file’s attribute to reflect 2:00 P.M. because the file’s attribute shows 1:00 P.M. and the in-memory Last Access Time shows 2:00 P.M.

Within a directory entry for a file

NTFS updates the directory entry for a file during the following events:

  • When NTFS updates the file’s Last Access Time and detects that the Last Access Time for the file differs by more than an hour from the Last Access Time stored in the file’s directory entry. This update typically occurs after a program closes the handle used to access a file within the directory. If the program holds the handle open for an extended time, a lag occurs before the change appears in the directory entry.
  • When NTFS updates other file attributes such as Last Modify Time, and a Last Access Time update is pending. In this case, NTFS updates the Last Access Time along with the other updates without additional performance impact.

Note: NTFS does not update a file’s directory entry when all in-memory references to that file are gone.

If you have an NTFS volume with a high number of folders or files, and a program is running that briefly accesses each of these in turn, the I/O bandwidth used to generate the Last Access Time updates can be a significant percentage of the overall I/O bandwidth.

Multiple Data Streams

A data stream is a sequence of bytes. An application populates the stream by writing data at specific offsets within the stream. The application can then read the data by reading the same offsets in the read path. Every file has a main, unnamed stream associated with it, regardless of the file system used.

However, NTFS supports additional named data streams in which each data stream is an alternate sequence of bytes as illustrated in the figure Unnamed and Named Streams. Applications can create additional named streams and access the streams by referring to their names. This feature permits related data to be managed as a single unit. For example, a graphics program can store a thumbnail image of bitmap in a named data stream within the NTFS file containing the image.

Unnamed and Named Streams

NTFS File System

FAT volumes support only the main, unnamed stream, so if you try to copy or move Streamexample.doc to a FAT volume or floppy disk, you receive an error message.

How NTFS File System Works: NTFS Physical Structure (5)

September 17th, 2009 Comments off

NTFS File Record Attributes

Every allocated sector on an NTFS volume belongs to a file. Even the file system metadata is part of a file. NTFS views each file (or folder) as a set of file attributes. File elements such as its name, its security information, and even its data are file attributes. Each attribute is identified by an attribute type code and an optional attribute name.

File and folder records are 1 KB each and are stored in the MFT, the attributes of which are written to the allocated space in the MFT. Besides file attributes, each file record contains information about the position of the file record in the MFT.

When a file’s attributes can fit within the MFT file record for that file, they are called resident attributes. Attributes such as file name and time stamp are always resident. When the amount of information for a file does not fit in its MFT file record, some file attributes become nonresident. Nonresident attributes are allocated one or more clusters of disk space. A portion of the nonresident attribute remains in the MFT and points to the external clusters. NTFS creates the Attribute List attribute to describe the location of all attribute records. The table NTFS File Attribute Types lists the file attributes currently defined by NTFS.

NTFS File Attribute Types

Attribute Type Description
Standard Information Information such as access mode (read-only, read/write, and so forth) timestamp, and link count.
Attribute List Locations of all attribute records that do not fit in the MFT record.
File Name A repeatable attribute for both long and short file names. The long name of the file can be up to 255 Unicode characters. The short name is the 8.3, case-insensitive name for the file. Additional names, or hard links, required by POSIX can be included as additional file name attributes.
Data File data. NTFS supports multiple data attributes per file. Each file typically has one unnamed data attribute. A file can also have one or more named data attributes.
Object ID A volume-unique file identifier. Used by the distributed link tracking service. Not all files have object identifiers.
Logged Tool Stream Similar to a data stream, but operations are logged to the NTFS log file just like NTFS metadata changes. This attribute is used by EFS.
Reparse Point Used for mounted drives. This is also used by Installable File System (IFS) filter drivers to mark certain files as special to that driver.
Index Root Used to implement folders and other indexes.
Index Allocation Used to implement the B-tree structure for large folders and other large indexes.
Bitmap Used to implement the B-tree structure for large folders and other large indexes.
Volume Information Used only in the $Volume system file. Contains the volume version.

NTFS creates a file record for each file and a folder record for each folder created on an NTFS volume. The MFT includes a separate file record for the MFT itself. These file and folder records are 1 KB each and are stored in the MFT. The attributes of the file are written to the allocated space in the MFT. Besides file attributes, each file record contains information about the position of the file record in the MFT. The figure MFT Entry with Resident Record shows the contents of an MFT record for a small file or folder. Small files and folders (typically, 900 bytes or smaller) are entirely contained within the file’s MFT record.

MFT Entry with Resident Record

image

Typically, each file uses one file record. However, if a file has a large number of attributes or becomes highly fragmented, it might need more than one file record. If this is the case, the first record for the file, the base file record, stores the location of the other file records required by the file.

Folder records contain index information. Small folder records reside entirely within the MFT structure, while large folders are organized B-tree structures and have records with pointers to external clusters that contain folder entries that cannot be contained within the MFT structure.

The benefit of using B-tree structures is evident when NTFS enumerates files in a large folder. The B-tree structure allows NTFS to group, or index, similar file names and then search only the group that contains the file, minimizing the number of disk accesses needed to find a particular file, especially for large folders. Because of the B-tree structure, NTFS outperforms FAT for large folders because FAT must scan all file names in a large folder before listing all of the files.

How NTFS File System Works: NTFS Physical Structure (4)

September 17th, 2009 Comments off

Master File Table

When you format a volume with NTFS, Windows Server 2003 creates an MFT and metadata files on the partition. The MFT is a relational database that consists of rows of file records and columns of file attributes. It contains at least one entry for every file on an NTFS volume, including the MFT itself.

The MFT stores the information required to retrieve files from the NTFS partition.

MFT and Metadata Files

Because the MFT stores information about itself, NTFS reserves the first 16 records of the MFT for metadata files (approximately 16 KB), which are used to describe the MFT. Metadata files that begin with a dollar sign ($) are described in the table Metadata Files Stored in the MFT. The remaining records of the MFT contain the file and folder records for each file and folder on the volume.

Metadata Files Stored in the MFT

System File File Name MFT Record Purpose of the File
Master file table $Mft 0 Contains one base file record for each file and folder on an NTFS volume. If the allocation information for a file or folder is too large to fit within a single record, other file records are allocated as well.
Master file table mirror $MftMirr 1 Guarantees access to the MFT in case of a single-sector failure. It is a duplicate image of the first four records of the MFT.
Log file $LogFile 2 Contains information used by NTFS for faster recoverability. The log file is used by Windows Server 2003 to restore metadata consistency to NTFS after a system failure. The size of the log file depends on the size of the volume, but you can increase the size of the log file by using the Chkdsk command.
Volume $Volume 3 Contains information about the volume, such as the volume label and the volume version.
Attribute definitions $AttrDef 4 Lists attribute names, numbers, and descriptions.
Root file name index . 5 The root folder.
Cluster bitmap $Bitmap 6 Represents the volume by showing free and unused clusters.
Boot sector $Boot 7 Includes the BPB used to mount the volume and additional bootstrap loader code used if the volume is bootable.
Bad cluster file $BadClus 8 Contains bad clusters for a volume.
Security file $Secure 9 Contains unique security descriptors for all files within a volume.
Upcase table $Upcase 10 Converts lowercase characters to matching Unicode uppercase characters.
NTFS extension file $Extend 11 Used for various optional extensions such as quotas, reparse point data, and object identifiers.
12–15 Reserved for future use.

The data segment locations for both the MFT and the backup MFT, $Mft and $MftMirr, respectively, are recorded in the boot sector. The $MftMirr is a duplicate image of either the first four records of the $Mft or the first cluster of the $Mft, whichever is larger. If any MFT records in the mirrored range are corrupted or unreadable, NTFS reads the boot sector to find the location of the $MftMirr. NTFS then reads the $MftMirr and uses the information in $MftMirr instead of the information in the MFT. If possible, the correct data from the $MftMirr is written back to the corresponding location in the $Mft.

MFT Zone

To prevent the MFT from becoming fragmented, NTFS reserves 12.5 percent of volume by default for exclusive use of the MFT. This space, known as the MFT zone, is not used to store data unless the remainder of the volume becomes full.

Depending on the average file size and other variables, as the volume fills to capacity, either the MFT zone or the unreserved space on the volume becomes full first.

  • Volumes that have a small number of large files exhaust the unreserved space first.
  • Volumes with a large number of small files exhaust the MFT zone space first.

In either case, fragmentation of the MFT occurs when one region or the other becomes full. You can change the size of the MFT zone for newly created volumes by to correspond to a percentage of the volume to be used as the MFT zone. The MFT zone sizes follow:

  • Setting 1, the default, reserves approximately 12.5 percent of the volume.
  • Setting 2 reserves approximately 25 percent.
  • Setting 3 reserves approximately 37.5 percent.
  • Setting 4 reserves approximately 50 percent.

In most computers, the default setting of 1 is adequate. The default setting accommodates volumes with an average file size of 8 KB. Storing a large number of smaller files might necessitate that you increase the size of the MFT zone for new volumes.

After you increase the size of the MFT zone, NTFS does not immediately allocate space to accommodate the size of the new MFT zone. Instead, NTFS exhausts the original reserved space before increasing the size of the MFT zone. When the original space is exhausted, NTFS looks for the next contiguous space large enough to hold the additional MFT zone, which can cause the MFT to become fragmented. You can adjust the zone size for the MFT if the defaults do not fit your needs.