| Definitions | |
| ~~~~~~~~~~~ | |
| Userspace filesystem: | |
| A filesystem in which data and metadata are provided by an ordinary | |
| userspace process. The filesystem can be accessed normally through | |
| the kernel interface. | |
| Filesystem daemon: | |
| The process(es) providing the data and metadata of the filesystem. | |
| Non-privileged mount (or user mount): | |
| A userspace filesystem mounted by a non-privileged (non-root) user. | |
| The filesystem daemon is running with the privileges of the mounting | |
| user. NOTE: this is not the same as mounts allowed with the "user" | |
| option in /etc/fstab, which is not discussed here. | |
| Filesystem connection: | |
| A connection between the filesystem daemon and the kernel. The | |
| connection exists until either the daemon dies, or the filesystem is | |
| umounted. Note that detaching (or lazy umounting) the filesystem | |
| does _not_ break the connection, in this case it will exist until | |
| the last reference to the filesystem is released. | |
| Mount owner: | |
| The user who does the mounting. | |
| User: | |
| The user who is performing filesystem operations. | |
| What is FUSE? | |
| ~~~~~~~~~~~~~ | |
| FUSE is a userspace filesystem framework. It consists of a kernel | |
| module (fuse.ko), a userspace library (libfuse.*) and a mount utility | |
| (fusermount). | |
| One of the most important features of FUSE is allowing secure, | |
| non-privileged mounts. This opens up new possibilities for the use of | |
| filesystems. A good example is sshfs: a secure network filesystem | |
| using the sftp protocol. | |
| The userspace library and utilities are available from the FUSE | |
| homepage: | |
| http://fuse.sourceforge.net/ | |
| Filesystem type | |
| ~~~~~~~~~~~~~~~ | |
| The filesystem type given to mount(2) can be one of the following: | |
| 'fuse' | |
| This is the usual way to mount a FUSE filesystem. The first | |
| argument of the mount system call may contain an arbitrary string, | |
| which is not interpreted by the kernel. | |
| 'fuseblk' | |
| The filesystem is block device based. The first argument of the | |
| mount system call is interpreted as the name of the device. | |
| Mount options | |
| ~~~~~~~~~~~~~ | |
| 'fd=N' | |
| The file descriptor to use for communication between the userspace | |
| filesystem and the kernel. The file descriptor must have been | |
| obtained by opening the FUSE device ('/dev/fuse'). | |
| 'rootmode=M' | |
| The file mode of the filesystem's root in octal representation. | |
| 'user_id=N' | |
| The numeric user id of the mount owner. | |
| 'group_id=N' | |
| The numeric group id of the mount owner. | |
| 'default_permissions' | |
| By default FUSE doesn't check file access permissions, the | |
| filesystem is free to implement its access policy or leave it to | |
| the underlying file access mechanism (e.g. in case of network | |
| filesystems). This option enables permission checking, restricting | |
| access based on file mode. It is usually useful together with the | |
| 'allow_other' mount option. | |
| 'allow_other' | |
| This option overrides the security measure restricting file access | |
| to the user mounting the filesystem. This option is by default only | |
| allowed to root, but this restriction can be removed with a | |
| (userspace) configuration option. | |
| 'max_read=N' | |
| With this option the maximum size of read operations can be set. | |
| The default is infinite. Note that the size of read requests is | |
| limited anyway to 32 pages (which is 128kbyte on i386). | |
| 'blksize=N' | |
| Set the block size for the filesystem. The default is 512. This | |
| option is only valid for 'fuseblk' type mounts. | |
| Control filesystem | |
| ~~~~~~~~~~~~~~~~~~ | |
| There's a control filesystem for FUSE, which can be mounted by: | |
| mount -t fusectl none /sys/fs/fuse/connections | |
| Mounting it under the '/sys/fs/fuse/connections' directory makes it | |
| backwards compatible with earlier versions. | |
| Under the fuse control filesystem each connection has a directory | |
| named by a unique number. | |
| For each connection the following files exist within this directory: | |
| 'waiting' | |
| The number of requests which are waiting to be transferred to | |
| userspace or being processed by the filesystem daemon. If there is | |
| no filesystem activity and 'waiting' is non-zero, then the | |
| filesystem is hung or deadlocked. | |
| 'abort' | |
| Writing anything into this file will abort the filesystem | |
| connection. This means that all waiting requests will be aborted an | |
| error returned for all aborted and new requests. | |
| Only the owner of the mount may read or write these files. | |
| Interrupting filesystem operations | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| If a process issuing a FUSE filesystem request is interrupted, the | |
| following will happen: | |
| 1) If the request is not yet sent to userspace AND the signal is | |
| fatal (SIGKILL or unhandled fatal signal), then the request is | |
| dequeued and returns immediately. | |
| 2) If the request is not yet sent to userspace AND the signal is not | |
| fatal, then an 'interrupted' flag is set for the request. When | |
| the request has been successfully transferred to userspace and | |
| this flag is set, an INTERRUPT request is queued. | |
| 3) If the request is already sent to userspace, then an INTERRUPT | |
| request is queued. | |
| INTERRUPT requests take precedence over other requests, so the | |
| userspace filesystem will receive queued INTERRUPTs before any others. | |
| The userspace filesystem may ignore the INTERRUPT requests entirely, | |
| or may honor them by sending a reply to the _original_ request, with | |
| the error set to EINTR. | |
| It is also possible that there's a race between processing the | |
| original request and its INTERRUPT request. There are two possibilities: | |
| 1) The INTERRUPT request is processed before the original request is | |
| processed | |
| 2) The INTERRUPT request is processed after the original request has | |
| been answered | |
| If the filesystem cannot find the original request, it should wait for | |
| some timeout and/or a number of new requests to arrive, after which it | |
| should reply to the INTERRUPT request with an EAGAIN error. In case | |
| 1) the INTERRUPT request will be requeued. In case 2) the INTERRUPT | |
| reply will be ignored. | |
| Aborting a filesystem connection | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| It is possible to get into certain situations where the filesystem is | |
| not responding. Reasons for this may be: | |
| a) Broken userspace filesystem implementation | |
| b) Network connection down | |
| c) Accidental deadlock | |
| d) Malicious deadlock | |
| (For more on c) and d) see later sections) | |
| In either of these cases it may be useful to abort the connection to | |
| the filesystem. There are several ways to do this: | |
| - Kill the filesystem daemon. Works in case of a) and b) | |
| - Kill the filesystem daemon and all users of the filesystem. Works | |
| in all cases except some malicious deadlocks | |
| - Use forced umount (umount -f). Works in all cases but only if | |
| filesystem is still attached (it hasn't been lazy unmounted) | |
| - Abort filesystem through the FUSE control filesystem. Most | |
| powerful method, always works. | |
| How do non-privileged mounts work? | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| Since the mount() system call is a privileged operation, a helper | |
| program (fusermount) is needed, which is installed setuid root. | |
| The implication of providing non-privileged mounts is that the mount | |
| owner must not be able to use this capability to compromise the | |
| system. Obvious requirements arising from this are: | |
| A) mount owner should not be able to get elevated privileges with the | |
| help of the mounted filesystem | |
| B) mount owner should not get illegitimate access to information from | |
| other users' and the super user's processes | |
| C) mount owner should not be able to induce undesired behavior in | |
| other users' or the super user's processes | |
| How are requirements fulfilled? | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| A) The mount owner could gain elevated privileges by either: | |
| 1) creating a filesystem containing a device file, then opening | |
| this device | |
| 2) creating a filesystem containing a suid or sgid application, | |
| then executing this application | |
| The solution is not to allow opening device files and ignore | |
| setuid and setgid bits when executing programs. To ensure this | |
| fusermount always adds "nosuid" and "nodev" to the mount options | |
| for non-privileged mounts. | |
| B) If another user is accessing files or directories in the | |
| filesystem, the filesystem daemon serving requests can record the | |
| exact sequence and timing of operations performed. This | |
| information is otherwise inaccessible to the mount owner, so this | |
| counts as an information leak. | |
| The solution to this problem will be presented in point 2) of C). | |
| C) There are several ways in which the mount owner can induce | |
| undesired behavior in other users' processes, such as: | |
| 1) mounting a filesystem over a file or directory which the mount | |
| owner could otherwise not be able to modify (or could only | |
| make limited modifications). | |
| This is solved in fusermount, by checking the access | |
| permissions on the mountpoint and only allowing the mount if | |
| the mount owner can do unlimited modification (has write | |
| access to the mountpoint, and mountpoint is not a "sticky" | |
| directory) | |
| 2) Even if 1) is solved the mount owner can change the behavior | |
| of other users' processes. | |
| i) It can slow down or indefinitely delay the execution of a | |
| filesystem operation creating a DoS against the user or the | |
| whole system. For example a suid application locking a | |
| system file, and then accessing a file on the mount owner's | |
| filesystem could be stopped, and thus causing the system | |
| file to be locked forever. | |
| ii) It can present files or directories of unlimited length, or | |
| directory structures of unlimited depth, possibly causing a | |
| system process to eat up diskspace, memory or other | |
| resources, again causing DoS. | |
| The solution to this as well as B) is not to allow processes | |
| to access the filesystem, which could otherwise not be | |
| monitored or manipulated by the mount owner. Since if the | |
| mount owner can ptrace a process, it can do all of the above | |
| without using a FUSE mount, the same criteria as used in | |
| ptrace can be used to check if a process is allowed to access | |
| the filesystem or not. | |
| Note that the ptrace check is not strictly necessary to | |
| prevent B/2/i, it is enough to check if mount owner has enough | |
| privilege to send signal to the process accessing the | |
| filesystem, since SIGSTOP can be used to get a similar effect. | |
| I think these limitations are unacceptable? | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| If a sysadmin trusts the users enough, or can ensure through other | |
| measures, that system processes will never enter non-privileged | |
| mounts, it can relax the last limitation with a "user_allow_other" | |
| config option. If this config option is set, the mounting user can | |
| add the "allow_other" mount option which disables the check for other | |
| users' processes. | |
| Kernel - userspace interface | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| The following diagram shows how a filesystem operation (in this | |
| example unlink) is performed in FUSE. | |
| NOTE: everything in this description is greatly simplified | |
| | "rm /mnt/fuse/file" | FUSE filesystem daemon | |
| | | | |
| | | >sys_read() | |
| | | >fuse_dev_read() | |
| | | >request_wait() | |
| | | [sleep on fc->waitq] | |
| | | | |
| | >sys_unlink() | | |
| | >fuse_unlink() | | |
| | [get request from | | |
| | fc->unused_list] | | |
| | >request_send() | | |
| | [queue req on fc->pending] | | |
| | [wake up fc->waitq] | [woken up] | |
| | >request_wait_answer() | | |
| | [sleep on req->waitq] | | |
| | | <request_wait() | |
| | | [remove req from fc->pending] | |
| | | [copy req to read buffer] | |
| | | [add req to fc->processing] | |
| | | <fuse_dev_read() | |
| | | <sys_read() | |
| | | | |
| | | [perform unlink] | |
| | | | |
| | | >sys_write() | |
| | | >fuse_dev_write() | |
| | | [look up req in fc->processing] | |
| | | [remove from fc->processing] | |
| | | [copy write buffer to req] | |
| | [woken up] | [wake up req->waitq] | |
| | | <fuse_dev_write() | |
| | | <sys_write() | |
| | <request_wait_answer() | | |
| | <request_send() | | |
| | [add request to | | |
| | fc->unused_list] | | |
| | <fuse_unlink() | | |
| | <sys_unlink() | | |
| There are a couple of ways in which to deadlock a FUSE filesystem. | |
| Since we are talking about unprivileged userspace programs, | |
| something must be done about these. | |
| Scenario 1 - Simple deadlock | |
| ----------------------------- | |
| | "rm /mnt/fuse/file" | FUSE filesystem daemon | |
| | | | |
| | >sys_unlink("/mnt/fuse/file") | | |
| | [acquire inode semaphore | | |
| | for "file"] | | |
| | >fuse_unlink() | | |
| | [sleep on req->waitq] | | |
| | | <sys_read() | |
| | | >sys_unlink("/mnt/fuse/file") | |
| | | [acquire inode semaphore | |
| | | for "file"] | |
| | | *DEADLOCK* | |
| The solution for this is to allow the filesystem to be aborted. | |
| Scenario 2 - Tricky deadlock | |
| ---------------------------- | |
| This one needs a carefully crafted filesystem. It's a variation on | |
| the above, only the call back to the filesystem is not explicit, | |
| but is caused by a pagefault. | |
| | Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2 | |
| | | | |
| | [fd = open("/mnt/fuse/file")] | [request served normally] | |
| | [mmap fd to 'addr'] | | |
| | [close fd] | [FLUSH triggers 'magic' flag] | |
| | [read a byte from addr] | | |
| | >do_page_fault() | | |
| | [find or create page] | | |
| | [lock page] | | |
| | >fuse_readpage() | | |
| | [queue READ request] | | |
| | [sleep on req->waitq] | | |
| | | [read request to buffer] | |
| | | [create reply header before addr] | |
| | | >sys_write(addr - headerlength) | |
| | | >fuse_dev_write() | |
| | | [look up req in fc->processing] | |
| | | [remove from fc->processing] | |
| | | [copy write buffer to req] | |
| | | >do_page_fault() | |
| | | [find or create page] | |
| | | [lock page] | |
| | | * DEADLOCK * | |
| Solution is basically the same as above. | |
| An additional problem is that while the write buffer is being copied | |
| to the request, the request must not be interrupted/aborted. This is | |
| because the destination address of the copy may not be valid after the | |
| request has returned. | |
| This is solved with doing the copy atomically, and allowing abort | |
| while the page(s) belonging to the write buffer are faulted with | |
| get_user_pages(). The 'req->locked' flag indicates when the copy is | |
| taking place, and abort is delayed until this flag is unset. |