source: orange/Orange/doc/modules/orngServerFiles.htm @ 9671:a7b056375472

Revision 9671:a7b056375472, 10.1 KB checked in by anze <anze.staric@…>, 2 years ago (diff)

Moved orange to Orange (part 2)

Line 
1<html>
2
3<head>
4<title>orngServerFiles: Orange's file repository</title>
5<link rel=stylesheet href="../style.css" type="text/css">
6<link rel=stylesheet href="style-print.css" type="text/css" media=print>
7</head>
8
9<body>
10<h1>orngServerFiles: Orange's file repository</h1>
11<index name="modules/orngServerFiles">
12<index name="orngServerFiles">
13<index name="files">
14
15<p>Module <code>orngServerFiles</code> allows users to download files from a common
16repository residing on the Orange server. It was designed to simplify the download and updates of
17external data sources for Orange Genomics add-on. Furthermore, an authenticated
18user can also manage the repository files with this module.</p>
19
20<p>Orange server file repository was created with the intend to store larger files that do not come with Orange installation, but
21may be required from the user when running specific Orange functions. A typical example is Orange bioinformatics package,
22which relies on large data files storing genome information. These do not come pre-installed, but are rather downloaded
23from the server when needed and stored in the local repository. Module orngServerFiles provides low-level functionality
24to manage these files, and is used by Orange modules to locate the files from the local repository and update/download them when
25and if needed.</p>
26
27<p>Each file managed by <code>orngServerFiles</code> is described by domain and
28the file name. Domains are like directories - a place where files are put in.</p>
29
30<h2>Local file management</h2>
31
32<p>Downloaded files are saved under Orange's settings directory, subdirectory
33<code>buffer/bigfiles</code>. For each new domain a subdirectory containing
34the downloaded files is created. With every download a corresponding
35info file is created with bearing the same name but added an extension plus &quot;.info&quot;.
36Info fail contains file's title, tags, size and date and time of the upload.
37</p>
38
39<p class="section">Methods used for managing local files</p>
40
41<dl class="attributes">
42<dt>download(domain, filename, serverfiles=None, callback=None, extract=True, verbose=True)</dt>
43<dd>Downloads file from the repository to local orange installation. To download
44files as an authenticated user you should also pass an instance of <code>ServerFiles</code> class.
45Callback can be a function without arguments. It will be called once for each downloaded percent of file: 100 times for the whole file.
46</dd>
47<dt>listdomains()</dt>
48<dd>List all file domains in the local repository.</dd>
49<dt>listfiles(domain)</dt>
50<dd>List all files from a domain in a local repository.</dd>
51<dt>info(domain, filename)</dt>
52<dd>Returns a dictionary containing downloaded file information. Keys: title, tags, size, datetime. </dd>
53<dt>allinfo(domain)</dt>
54<dd>Goes through all files in a domain on a local repository and returns a dictionary, where keys are names of the files and values are their information.</dd>
55<dt>search(substrings, caseSensitive=False, inTag=True, inTitle=True, inName=True)</dt>
56<dd>Function searches for files in the local repository where all substrings in a list are contained
57in at least one chosen field (tag, title, name). It returns a list of tuples: first tuple element
58is the domain of the file, second its name.</dd> 
59<dt>localpath(domain [,filename])</dt>
60<dd>Returns a path for the domain in the local repository. If argument <code>filename</code> is given, returns a path to corresponding file.</dd>
61<dt>localpath_download(domain, filename, **kwargs)</dt>
62<dd>Returns local path for the given domain and file. If file does not exist on the local machine, download it. Additional arguments are passed to the <code>download</code> function.</dd>
63<dt>remove_domain(domain [,force])</dt>
64<dd>Removes a domain. If force is True, domain is removed even
65if it is not empty (contains files).</dd>
66<dt>remove(domain, filename)</dt>
67<dd>Removes a file from repository.</dd>
68<dt>update(domain, filename, access_code=None, verbose=True)</dt>
69<dd>Downloads the corresponding file from the server and places it in the local repository, but only if the server copy of the file is newer or the local copy does not exist. An optional <code>access_code</code> can be given to allow server access if the file is protected.</dd>
70
71
72</dl>
73
74<h2>ServerFiles: the repository</h2>
75
76<p>To work with the repository, you need to create an instance of
77<code>ServerFiles</code> object. To access the repository as an authenticated user,
78a username and password should be passed to the constructor.
79All password
80protected operations and transfers are secured by SSL; this secures both password and content.</p>
81
82<p>Repository files are set as protected when first uploaded:
83only authenticated users can see them. They need to be unprotected
84for public use.</p>
85
86<dl class="attributes">
87<dt>ServerFiles(username=None, password=None, access_code=None)</dt>
88<dd>Creates a ServerFiles instance. Pass your username and password to use
89the repository as an authenticated user. If you want to use your access code
90(as an non-authenticated user), pass it also.</dd>
91</dl>
92
93<p class="section">Non-authenticated users</p>
94
95<dl class="attributes">
96<dt>download(domain, filename, target, callback=None)</dt>
97<dd>Downloads file from the repository to a given target name. Callback can be a function without arguments. It will be called once for each downloaded percent of file: 100 times for the whole file.
98</dd>
99<dt>listdomains()</dt>
100<dd>List all domains on repository.</dd>
101<dt>listfiles(domain)</dt>
102<dd>List all files in a repository domain.</dd>
103<dt>info(domain, filename)</dt>
104<dd>Returns a dictionary containing repository file info. Keys: title, tags, size, datetime.</dd>
105<dt>allinfo(domain)</dt>
106<dd>Goes through all accessible files in a given domain and returns a dictionary, where key
107is file's name and value its info.</dd>
108<dt>search(substrings, caseSensitive=False, inTag=True, inTitle=True, inName=True)</dt>
109<dd>Function searches for files on the repository where all substrings in a list are contained
110in at least one choosen field (tag, title, name). It returns a list of tuples: first tuple element
111is the file's domain, second its name. As for now the search is performed locally, therefore
112information on files in repository is transfered on first call of this function. </dd> 
113<dt>downloadFH(domain, filename)</dt>
114<dd>Returns a file handle to the file that we would like to download.
115</dd>
116</dl>
117
118<p class="section">Authenticated users</p>
119
120Authenticated users can use methods described in &quot;Non-authenticated users&quot;
121section, but they can also use the functions that access and modify protected files.
122
123<dl class="attributes">
124<dt>create_domain(domain)</dt>
125<dd>Creates a repository domain.</dd>
126<dt>remove_domain(domain [,force])</dt>
127<dd>Removes a domain. If force is True, domain is removed even
128if it is not empty (contains files).</dd>
129<dt>remove(domain, filename)</dt>
130<dd>Removes a file from the repository.</dd>
131<dt>protection(domain, filename)</dt>
132<dd>Returns file protection. Legend: &quot;0&quot; - public use, &quot;1&quot; - for authenticated
133users only, anything else represents a specific access code.</dd>
134<dt>protect(domain, filename [, access_code])</dt>
135<dd>Hide file from non-authenticated users. If an access code (string) is passed, the file
136will be available to authenticated users and non-authenticated users with that access
137code.</dd>
138<dt>unprotect(domain, filename)</dt>
139<dd>Put a file into public use.</dd>
140<dt>upload(domain, filename, localfile [, title, tags])</dt>
141<dd>Uploads a file &quot;localfile&quot; to the domain where it is saved with
142filename &quot;filename&quot;. If file does not exist yet, set it as
143protected. Parameter localfile can be a file handle open for reading or
144a file name.</dd>
145</dl>
146
147<h2>Examples</h2>
148
149<h3>Downloading and listing files</h3>
150
151<p>Listing local files, files from the repository and downloading all
152available files from domain &quot;demo&quot;.</p>
153
154<xmp class=code>import orngServerFiles
155
156repository = orngServerFiles.ServerFiles()
157print "My files", orngServerFiles.listfiles('demo')
158print "Repository files", repository.listfiles('demo')
159
160print "Downloading all files in domain 'test'"
161for file in repository.listfiles('demo'):
162    print "Datetime for", file, repository.info('demo', file)["datetime"]
163    orngServerFiles.download('demo', file)
164
165print "My files after download", orngServerFiles.listfiles('demo')
166print "My domains", orngServerFiles.listdomains()
167</xmp>
168
169<p>Output for first run (and current repository state):</p>
170
171<xmp class=code>My files []
172Repository files ['orngServerFiles.py', 'urllib2_file.py']
173Downloading all files in domain 'test'
174Datetime for orngServerFiles.py 2008-08-20 12:25:54.624000
175Datetime for urllib2_file.py 2008-08-20 12:25:54.827000
176My files after download ['urllib2_file.py', 'orngServerFiles.py']
177My domains ['demo']
178</xmp>
179
180<h3>Creating a domain, uploading files</h3>
181
182A similar domain as &quot;demo&quot; in previous example can be built as follows.
183
184<xmp class=code>import orngServerFiles
185
186ordinary = orngServerFiles.ServerFiles()
187authenticated = orngServerFiles.ServerFiles(username, password)
188
189try:
190    authenticated.remove_domain('demo2', force=True)
191except:
192    pass
193
194authenticated.create_domain('demo2')
195
196authenticated.upload('demo2', 'orngServerFiles.py', 'orngServerFiles.py', \
197    title="orngServerFiles client", tags=["basic", "fileManagement"])
198authenticated.upload('demo2', 'urllib2_file.py', 'urllib2_file.py')
199print "Non-authenticated users see:", ordinary.listfiles('demo2')
200print "Authenticated users see:", authenticated.listfiles('demo2')
201authenticated.unprotect('demo2', 'orngServerFiles.py')
202authenticated.unprotect('demo2', 'urllib2_file.py')
203print "Non-authenticated users now see:", ordinary.listfiles('demo2')
204print "orngServerFiles.py file info:"
205import pprint; pprint.pprint(ordinary.info('demo2', 'orngServerFiles.py'))
206</xmp>
207
208<p>Output:</p>
209
210<xmp class=code>Ordinary users see: ['']
211Authenticated users see: ['orngServerFiles.py', 'urllib2_file.py']
212Non-authenticated users now see: ['orngServerFiles.py', 'urllib2_file.py']
213orngServerFiles.py file info:
214{'datetime': '2008-08-26 10:30:05.373000',
215 'size': '10165',
216 'tags': ['basic', 'fileManagement'],
217 'title': 'orngServerFiles client'}
218</xmp>
219
220</body>
221</html>
222
Note: See TracBrowser for help on using the repository browser.