Writing flexible filesystems with
FUSE-Python
Also available at
http://xinh.org/fuse-python
Talk overview
- UNIX based filesystems
- What's FUSE?
- FUSE API overview
- ToyFS
- Q&A
On filesystems
“One of the real contributions of UNIX has been the view that everything is a file.”
Filesystems
| FS | Platform |
|---|---|
| vfat | Windows |
| hfs+ | OS X |
| ext4 | Linux |
| xfs | IRIX, Linux |
| nfs | Linux* |
| iso9660 | CD-ROM |
| fuse | Linux* |
# cat /proc/filesystems
Files
| Symbol | Meaning |
|---|---|
| d | Directory |
| l | Symbolic link |
| c | Character device |
| b | Block device |
| s | Socket |
| p | Named pipe |
| - | Regular file |
FUSE
Filesystem in USErspace
Virtual memory segregation
Why FUSE?
- Usable by unprivileged users
- Easier development cycle
- Easy to install (apt-get install sshfs)
- Multiple language support and bindings
- C, C++, Python, Java, Ruby, Perl, Golang, Lua
- Ported to FreeBSD, Mac OSX, OpenSolaris
FUSE overview
How do I install FUSE?
# yum install fuse fuse-python
# apt-get install fuse python-fuse
Loading fuse
# modprobe fuse
# lsmod | grep fuse
Filesystem class
Subclass fuse.Fuse class and implement a number of methods.
Mount filesystem
Instantiate the Filesystem class and call main()
mount
$ python toyfs.py /tmp/toy
getattr(path)
Defining this method is mandatory for a working filesystem.
The stat structure
| Member | Description |
|---|---|
| st_mode | Inode protection mode |
| st_ino | File serial number |
| st_dev | Device ID |
| st_nlink | Number of hard links |
| st_uid | User ID of file |
| st_gid | Group ID of file |
| st_size | File size in bytes |
| st_atime | Time of last access |
| st_mtime | Time of last data modification |
| st_ctime | Time of last status change |
ref: <sys/stat.h>
$ stat /etc/fstab
[anurag@zomg toyfs]$ stat /etc/fstab File: ‘/etc/fstab’ Size: 481 Blocks: 8 IO Block: 4096 regular file Device: fd01h/64769d Inode: 259076 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: system_u:object_r:etc_t:s0 Access: 2015-02-20 14:46:16.248920273 +0530 Modify: 2015-02-11 20:42:51.750210844 +0530 Change: 2015-02-11 21:56:52.520767293 +0530 Birth: - [anurag@zomg toyfs]$
st_mode
st.st_mode = stat.S_IFDIR | 0755
Each bit of the output is 0 if the corresponding bit of x AND of y is 0, otherwise it's 1
ST_MODE flags
| Type | Flag |
|---|---|
| d | stat.S_IFDIR |
| l | stat.S_IFLNK |
| c | stat.S_IFCHR |
| b | stat.S_IFBLK |
| s | stat.S_IFSOCK |
| p | stat.S_IFIFO |
| - | stat.S_IFREG |
ref: Python stat module
readdir(path)
Read directory contents
Reading files
open(path, flags)
read(path, length, offset)
Filesystem methods
| General | File operation |
|---|---|
| getattr(path) | open(path, flags) |
| mkdir(path, mode) | create(path, flags, mode) |
| rename(old, new) | read(path, length, offset) |
| mknod(path, mode, rdev) | write(path, buf, offset) |
| link(target, name) | fgetattr(path) |
| symlink(target, name) | ftruncate(path, len) |
| readlink(path) | flush(path) |
| unlink(path) | release(path) |
| fsinit(self) | fsync(path, fdatasync) |
FUSE filesystems
fuse-zip, rarfs, mysqlfs, cryptfs, httpfs, sshfs, imapfs, gmailfs, flickrfs, ntfs-3g, gitfs, and many many more...