MongoDB: 10. Grid FS

GridFS 的文件同樣是保存在 db.collection 中，通常使用 fs.files 存儲文件元數(shù)據(jù)信息，fs.chunks 存儲文件內(nèi)容。

存儲海量文件，得啟用 Auto-Sharding。

> admin.runCommand({ enablesharding: test })
> admin.runCommand({ shardcollection: test.fs.chunks, key:{files_id:1} })
PyMongo API 中有個 GridFS 對象，使用方法很簡單。

>>> from pymongo import *
>>> from pymongo.objectid import ObjectId
>>> from gridfs import *
>>> from pprint import pprint

>>> conn = Connection()
>>> db = conn.test
>>> gfs = GridFS(db)
隨便找個文件存到 GridFS 中，除了必須的 filename 外，還可以附加任意屬性。

>>> with open("/home/yuhen/a.txt", "r") as file:
... id = gfs.put(file.read(), filename = "/xxx/xxx/a.txt", upload = "q.yuhen", abc = 123)
... print id
...
4c85e1f8499b144773000000

>>> gfs.list()
[u'/xxx/xxx/a.txt']
用 mongo 看看數(shù)據(jù)庫中存儲的具體信息。

$ ./mongo

MongoDB shell version: 1.6.2
connecting to: test

> show collections
fs.chunks
fs.files
system.indexes

> db.fs.files.find()
{
"_id" : ObjectId("4c85e1f8499b144773000000"),
"abc" : 123,
"chunkSize" : 262144,
"upload" : "q.yuhen",
"filename" : "/xxx/xxx/a.txt",
"length" : 14,
"uploadDate" : "Tue Sep 07 2010 14:55:52 GMT+0800 (CST)",
"md5" : "284d1d15a9d681b288cb5915ada39f53"
}

> db.fs.chunks.find()
{
"_id" : ObjectId("4c85e1f8499b144773000001"),
"n" : 0,
"data" : BinData(2,"DgAAAGFiY2VmZywgaGVsbG8K"),
"files_id" : ObjectId("4c85e1f8499b144773000000")
}
我們可以用 file_id 讀取文件數(shù)據(jù)(包括文件內(nèi)容和附加屬性)。

>>> with open("/home/yuhen/a2.txt", "w") as file:
... out = gfs.get(ObjectId("4c85e1f8499b144773000000"))
... file.write(out.read())
... pprint(dir(out))
... pprint(out._file.items())
... print out.name
... print out.upload
... print out.abc
...

[
...,
'_file',
'_id',
'aliases',
'chunk_size',
'content_type',
'length',
'md5',
'metadata',
'name',
'read',
'seek',
'tell',
'upload_date'
]

[
(u'abc', 123),
(u'chunkSize', 262144),
(u'upload', u'q.yuhen'),
(u'filename', u'/xxx/xxx/a.txt'),
(u'length', 14),
(u'uploadDate', datetime.datetime(2010, 9, 7, 6, 55, 52, 611000)),
(u'_id', ObjectId('4c85e1f8499b144773000000')),
(u'md5', u'284d1d15a9d681b288cb5915ada39f53')
]

/xxx/xxx/a.txt
q.yuhen
123
還可以用 filename 讀取文件。在應(yīng)用中我們通常要確保 filename 唯一性。

>>> with open("/home/yuhen/a3.txt", "w") as file:
... out = gfs.get_last_version("/xxx/xxx/a.txt")
... file.write(out.read())
... print out.name, out.length
...

/xxx/xxx/a.txt 14
用相同的 filename 存儲同一文件的不同版本是允許的，數(shù)據(jù)庫中會保留所有歷史數(shù)據(jù)。但用 get_last_version(filename) 只能取回最后一次的更新數(shù)據(jù)，可以直接從 db.fs.files 中查詢 file_id 來獲取不同版本內(nèi)容。

>>> with open("/home/yuhen/a.txt", "r") as file:
... id = gfs.put(file.read(), filename = "/xxx/xxx/a.txt", upload = "q.yuhen", abc = 456)
... print id
...
4c85e70a499b144773000004

>>> gfs.list()
[u'/xxx/xxx/a.txt']

>>> with open("/home/yuhen/a3.txt", "w") as file:
... out = gfs.get_last_version("/xxx/xxx/a.txt")
... file.write(out.read())
... print out.name, out.length, out.abc
...
/xxx/xxx/a.txt 22 456

>>> for f in db.fs.files.find({ "filename":"/xxx/xxx/a.txt" }):
... print f["_id"], f["length"], f["abc"]
...
4c85e70a499b144773000004 22 456
4c85e1f8499b144773000000 14 123
GridFS 還提供了 exists、delete 等方法。delete 按 file_id 刪除，也就是說是文件的最后一個版本。

>>> def clear():
... for f in gfs.list():
... id = gfs.get_last_version(f)._id
... print id
... if gfs.exists(id): gfs.delete(id)
...

>>> clear()
4c85e70a499b144773000004

>>> gfs.list()
[u'/xxx/xxx/a.txt']

>>> clear()
4c85e1f8499b144773000000

>>> gfs.list()
[]