Document storage and security

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Document storage and security

Patrick Larkin-3
Hello -

I’ve been tasked with building an application which permits our employees to upload PDF documents of certain clearances. By nature, these document could contain sensitive information.  

I have used Lasso 8 to build uploaders before that simply upload a document to a designated folder,  rename it to something, and add a database entry for the file’s existence.  

in this case, I cannot use a standard naming convention.  Even if I encrypted the file name to contain the particulars needed to identify the file, I’m afraid that people will try different combinations of characters to try and find one.  In other words, the directory containing the files will be exposed.  

Am I overly concerned?  Is there a way to secure this so that only the person submitting the files can “see” them (aside from the administrator or the system that needs rights to everything)?

Anyone do anything similar?


PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Tim Taplin
One of the things you can do is make sure the files are stored outside the webroot, so that you can use Lasso to control access to the files. No one can just guess at a URL.

you can then implement whatever level of security that you need in the file or files that you use to serve up the documents.

Tim Taplin

> On Feb 27, 2015, at 1:05 PM, Patrick Larkin <[hidden email]> wrote:
>
> Hello -
>
> I’ve been tasked with building an application which permits our employees to upload PDF documents of certain clearances. By nature, these document could contain sensitive information.  
>
> I have used Lasso 8 to build uploaders before that simply upload a document to a designated folder,  rename it to something, and add a database entry for the file’s existence.  
>
> in this case, I cannot use a standard naming convention.  Even if I encrypted the file name to contain the particulars needed to identify the file, I’m afraid that people will try different combinations of characters to try and find one.  In other words, the directory containing the files will be exposed.  
>
> Am I overly concerned?  Is there a way to secure this so that only the person submitting the files can “see” them (aside from the administrator or the system that needs rights to everything)?
>
> Anyone do anything similar?
>
>
> PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
> ——
> Patrick Larkin
> Application Management Group
> Information Technology
> Bethlehem Area School District
> https://www.beth.k12.pa.us
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

RE: Document storage and security

Rick Draper-2
In reply to this post by Patrick Larkin-3
Hi Patrick,

You can store the files in a directory that is not directly accessible to
the Internet and use Lasso to serve them, based on privileges for the
operator.  If you want to allow only the operator who uploaded it to see it,
you would simply record the operator id in the file upload table where the
path for serving is also stored.

VBR

Rick

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Patrick Larkin
Sent: Saturday, 28 February 2015 6:06 AM
To: [hidden email]
Subject: Document storage and security

Hello -

I've been tasked with building an application which permits our employees to
upload PDF documents of certain clearances. By nature, these document could
contain sensitive information.  

I have used Lasso 8 to build uploaders before that simply upload a document
to a designated folder,  rename it to something, and add a database entry
for the file's existence.  

in this case, I cannot use a standard naming convention.  Even if I
encrypted the file name to contain the particulars needed to identify the
file, I'm afraid that people will try different combinations of characters
to try and find one.  In other words, the directory containing the files
will be exposed.  

Am I overly concerned?  Is there a way to secure this so that only the
person submitting the files can "see" them (aside from the administrator or
the system that needs rights to everything)?

Anyone do anything similar?


PLEASE NOTE:  My e-mail address has changed to:  [hidden email] --
Patrick Larkin Application Management Group Information Technology Bethlehem
Area School District https://www.beth.k12.pa.us


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email] Official list archives
available at http://www.lassotalk.com To unsubscribe, E-mail to:
<[hidden email]>
Send administrative queries to  <[hidden email]>




#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Patrick Larkin-3
In reply to this post by Tim Taplin
I thought about this but isn’t that directory still accessible if you tell Lasso it should be accessible?

PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us

On Feb 27, 2015, at 3:27 PM, Tim Taplin <[hidden email]> wrote:

> One of the things you can do is make sure the files are stored outside the webroot, so that you can use Lasso to control access to the files. No one can just guess at a URL.
>
> you can then implement whatever level of security that you need in the file or files that you use to serve up the documents.
>
> Tim Taplin
>
>> On Feb 27, 2015, at 1:05 PM, Patrick Larkin <[hidden email]> wrote:
>>
>> Hello -
>>
>> I’ve been tasked with building an application which permits our employees to upload PDF documents of certain clearances. By nature, these document could contain sensitive information.  
>>
>> I have used Lasso 8 to build uploaders before that simply upload a document to a designated folder,  rename it to something, and add a database entry for the file’s existence.  
>>
>> in this case, I cannot use a standard naming convention.  Even if I encrypted the file name to contain the particulars needed to identify the file, I’m afraid that people will try different combinations of characters to try and find one.  In other words, the directory containing the files will be exposed.  
>>
>> Am I overly concerned?  Is there a way to secure this so that only the person submitting the files can “see” them (aside from the administrator or the system that needs rights to everything)?
>>
>> Anyone do anything similar?
>>
>>
>> PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
>> ——
>> Patrick Larkin
>> Application Management Group
>> Information Technology
>> Bethlehem Area School District
>> https://www.beth.k12.pa.us
>>
>>
>> #############################################################
>>
>> This message is sent to you because you are subscribed to
>> the mailing list Lasso [hidden email]
>> Official list archives available at http://www.lassotalk.com
>> To unsubscribe, E-mail to: <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

stevepiercy
In reply to this post by Patrick Larkin-3
Create a directory structure, something like this.

/users/user1
...
/users/userN

Configure Apache to deny access to the directory /users/.

Store the username in the session data.

Store a reference the filename in a database, and serve it up
like this:

<a href="/filepath/download.lasso?file=my.pdf">my.pdf</a>

In download.lasso, grab the username from the session data (else
redirect to error.lasso because it is not an authenticated
user), then do a file_serve:

     file_serve(include_raw($filepath), -file=$filename);

You can also include the MIME type as an argument to file_serve.

--steve


On 2/27/15 at 3:05 PM, [hidden email] (Patrick Larkin) pronounced:

>Hello -
>
>I’ve been tasked with building an application which permits
>our employees to upload PDF documents of certain clearances. By
>nature, these document could contain sensitive information.
>I have used Lasso 8 to build uploaders before that simply
>upload a document to a designated folder,  rename it to
>something, and add a database entry for the file’s existence.
>in this case, I cannot use a standard naming convention.  Even
>if I encrypted the file name to contain the particulars needed
>to identify the file, I’m afraid that people will try
>different combinations of characters to try and find one.  In
>other words, the directory containing the files will be exposed.
>Am I overly concerned?  Is there a way to secure this so that
>only the person submitting the files can “see” them (aside
>from the administrator or the system that needs rights to everything)?
>
>Anyone do anything similar?
>
>
>PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
>——
>Patrick Larkin
>Application Management Group
>Information Technology
>Bethlehem Area School District
>https://www.beth.k12.pa.us
>
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Bil Corry-3
In reply to this post by Patrick Larkin-3
If the files are being shared with specific people, then you can use
authentication to only show them to people who authenticate (which is what
all the responses to this thread are suggesting).

If the files are being shared anonymously, and you're relying on a random
token to access the file, then yes, you should be concerned about someone
brute forcing the random token.  In that case, you can add controls to
mitigate (but not completely remediate) the issue.  Add brute forcing
detection, such as one IP is limited to 10 requests.  And make sure the
random token is a cryptographically strong GUID.  And you might add in
expiration for the document and/or allow unpublishing them once they've
been viewed, or haven't been viewed in X days, etc.


- Bil


On Fri, Feb 27, 2015 at 9:05 PM, Patrick Larkin <[hidden email]>
wrote:

> Hello -
>
> I’ve been tasked with building an application which permits our employees
> to upload PDF documents of certain clearances. By nature, these document
> could contain sensitive information.
>
> I have used Lasso 8 to build uploaders before that simply upload a
> document to a designated folder,  rename it to something, and add a
> database entry for the file’s existence.
>
> in this case, I cannot use a standard naming convention.  Even if I
> encrypted the file name to contain the particulars needed to identify the
> file, I’m afraid that people will try different combinations of characters
> to try and find one.  In other words, the directory containing the files
> will be exposed.
>
> Am I overly concerned?  Is there a way to secure this so that only the
> person submitting the files can “see” them (aside from the administrator or
> the system that needs rights to everything)?
>
> Anyone do anything similar?
>
>
> PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
> ——
> Patrick Larkin
> Application Management Group
> Information Technology
> Bethlehem Area School District
> https://www.beth.k12.pa.us
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Brian K. Middendorf-2
In reply to this post by Patrick Larkin-3
> On Feb 27, 2015, at 12:05 PM, Patrick Larkin <[hidden email]> wrote:
>
> in this case, I cannot use a standard naming convention.  Even if I encrypted the file name to contain the particulars needed to identify the file, I’m afraid that people will try different combinations of characters to try and find one.  In other words, the directory containing the files will be exposed.  
>
> Am I overly concerned?  Is there a way to secure this so that only the person submitting the files can “see” them (aside from the administrator or the system that needs rights to everything)?
>
> Anyone do anything similar?

Oer concern is a good thing.

I advise you to not return a file name clash with "that is already taken" or the like.  Stupid simple security through obscurity.  Much like locks on a door.

And, by all means, please, please, please do not do that on a login page!  I witness that far too often.  I usually take the time to send a polite email explainiing why that is a poor practice,

As always, YMMV.

-brian.




#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Patrick Larkin-3
In reply to this post by stevepiercy
Thanks for this.

I’m guessing Lasso can still server a file even though Apache cannot “see” the directory?

Also, would just placing the directory completely outside then web root and configuring Apache to see it achieve the same thing?  

I’m not sure how I’m going to name the files yet - it should be identifiable by a database record at the minimum but without a filename with an obvious naming convention.  

Once the directory is populated by 6000+ files, I’m hoping Lasso’s FILE tags are fast enough to parse through the directory and find the needed files upon request.  :)

(Making a directory for each user is tricky since that number fluctuates constantly and is over 2000 users.  Active Directory could help this I suppose.)


PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us

On Feb 27, 2015, at 3:31 PM, Steve Piercy - Website Builder <[hidden email]> wrote:

> Create a directory structure, something like this.
>
> /users/user1
> ...
> /users/userN
>
> Configure Apache to deny access to the directory /users/.
>
> Store the username in the session data.
>
> Store a reference the filename in a database, and serve it up like this:
>
> <a href="/filepath/download.lasso?file=my.pdf">my.pdf</a>
>
> In download.lasso, grab the username from the session data (else redirect to error.lasso because it is not an authenticated user), then do a file_serve:
>
>    file_serve(include_raw($filepath), -file=$filename);
>
> You can also include the MIME type as an argument to file_serve.
>
> --steve
>
>
> On 2/27/15 at 3:05 PM, [hidden email] (Patrick Larkin) pronounced:
>
>> Hello -
>>
>> I’ve been tasked with building an application which permits our employees to upload PDF documents of certain clearances. By nature, these document could contain sensitive information.
>> I have used Lasso 8 to build uploaders before that simply upload a document to a designated folder,  rename it to something, and add a database entry for the file’s existence.
>> in this case, I cannot use a standard naming convention.  Even if I encrypted the file name to contain the particulars needed to identify the file, I’m afraid that people will try different combinations of characters to try and find one.  In other words, the directory containing the files will be exposed.
>> Am I overly concerned?  Is there a way to secure this so that only the person submitting the files can “see” them (aside from the administrator or the system that needs rights to everything)?
>>
>> Anyone do anything similar?
>>
>>
>> PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
>> ——
>> Patrick Larkin
>> Application Management Group
>> Information Technology
>> Bethlehem Area School District
>> https://www.beth.k12.pa.us
>>
>>
>> #############################################################
>>
>> This message is sent to you because you are subscribed to
>> the mailing list Lasso [hidden email]
>> Official list archives available at http://www.lassotalk.com
>> To unsubscribe, E-mail to: <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Steve Piercy              Website Builder              Soquel, CA
> <[hidden email]>               <http://www.StevePiercy.com/>
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
> the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
2 mar 2015 kl. 14:43 skrev Patrick Larkin <[hidden email]>:

> I’m not sure how I’m going to name the files yet - it should be identifiable by a database record at the minimum but without a filename with an obvious naming convention.  

I suggest using the id of the record the file meta data is stored in. In our case that is a GUID.
Files will then have names like 1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg

> Once the directory is populated by 6000+ files, I’m hoping Lasso’s FILE tags are fast enough to parse through the directory and find the needed files upon request.  :)

We use a directory per date the file was uploaded. The path to the dir is then stored together with the file name as stored on disk, and the original name grabbed when uploaded. That way there are never too many files in one directory. A directory path can look like this:
/_files/userpics/20150302/
The full path to the file would be:
/_files/userpics/20150302/1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg

After validating that this is a legit user files are served:
web_response -> sendfile(file_read(#stored_path + #stored_filename), #original_filename, -type = #mime_type, -disposition = #disposition)

With this approach files are served fast, to valid users only, never exposing the stored name or path, and using the original name when file is saved on the users disk.

(This is of course Lasso 9 code. Same approach can be done in Lasso 8.)

Oh, and we use a URL scene that makes the URL look like a proper file link.
https://mysite.tld/file/MUZFNzQ4QUMtNDI4Qy00RThCLThDMzEtQkEwNUExNzNDODg2/original_file_name.jpg

The onefile solution grabs the GUID only and skips the rest, but it makes the Url look nicer and less scary.
(Yes, the GUID is masked in the URL)

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
In reply to this post by Patrick Larkin-3
2 mar 2015 kl. 14:43 skrev Patrick Larkin <[hidden email]>:

> I’m not sure how I’m going to name the files yet - it should be identifiable by a database record at the minimum but without a filename with an obvious naming convention.  

I suggest using the id of the record the file meta data is stored in. In our case that is a GUID.
Files will then have names like 1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg

> Once the directory is populated by 6000+ files, I’m hoping Lasso’s FILE tags are fast enough to parse through the directory and find the needed files upon request.  :)

We use a directory per date the file was uploaded. The path to the dir is then stored together with the file name as stored on disk, and the original name grabbed when uploaded. That way there are never too many files in one directory. A directory path can look like this:
/_files/userpics/20150302/
The full path to the file would be:
/_files/userpics/20150302/1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg

After validating that this is a legit user files are served:
web_response -> sendfile(file_read(#stored_path + #stored_filename), #original_filename, -type = #mime_type, -disposition = #disposition)

With this approach files are served fast, to valid users only, never exposing the stored name or path, and using the original name when file is saved on the users disk.

(This is of course Lasso 9 code. Same approach can be done in Lasso 8.)

Oh, and we use a URL scene that makes the URL look like a proper file link.
https://mysite.tld/file/MUZFNzQ4QUMtNDI4Qy00RThCLThDMzEtQkEwNUExNzNDODg2/original_file_name.jpg

The onefile solution grabs the GUID only and skips the rest, but it makes the Url look nicer and less scary.
(Yes, the GUID is masked in the URL)

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Patrick Larkin-3
In reply to this post by Jolle Carlestam-2
How do you make that crazy GUID?

And are you creating dated directories on demand with Lasso?

PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us

On Mar 2, 2015, at 9:16 AM, Jolle Carlestam <[hidden email]> wrote:

> 2 mar 2015 kl. 14:43 skrev Patrick Larkin <[hidden email]>:
>
>> I’m not sure how I’m going to name the files yet - it should be identifiable by a database record at the minimum but without a filename with an obvious naming convention.  
>
> I suggest using the id of the record the file meta data is stored in. In our case that is a GUID.
> Files will then have names like 1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg
>
>> Once the directory is populated by 6000+ files, I’m hoping Lasso’s FILE tags are fast enough to parse through the directory and find the needed files upon request.  :)
>
> We use a directory per date the file was uploaded. The path to the dir is then stored together with the file name as stored on disk, and the original name grabbed when uploaded. That way there are never too many files in one directory. A directory path can look like this:
> /_files/userpics/20150302/
> The full path to the file would be:
> /_files/userpics/20150302/1FE748AC-428C-4E8B-8C31-BA05A173C886.jpg
>
> After validating that this is a legit user files are served:
> web_response -> sendfile(file_read(#stored_path + #stored_filename), #original_filename, -type = #mime_type, -disposition = #disposition)
>
> With this approach files are served fast, to valid users only, never exposing the stored name or path, and using the original name when file is saved on the users disk.
>
> (This is of course Lasso 9 code. Same approach can be done in Lasso 8.)
>
> Oh, and we use a URL scene that makes the URL look like a proper file link.
> https://mysite.tld/file/MUZFNzQ4QUMtNDI4Qy00RThCLThDMzEtQkEwNUExNzNDODg2/original_file_name.jpg
>
> The onefile solution grabs the GUID only and skips the rest, but it makes the Url look nicer and less scary.
> (Yes, the GUID is masked in the URL)
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
2 mar 2015 kl. 16:16 skrev Patrick Larkin <[hidden email]>:

> How do you make that crazy GUID?

It’s not crazy, it is standard.
http://en.wikipedia.org/wiki/Globally_unique_identifier

And in Lasso 9 it’s easy:
lasso_uniqueid
-> FBEBE980-FC1A-471B-B8B3-AB593FB65D5E

>
> And are you creating dated directories on demand with Lasso?

Yes, again with Lasso 9 it is very easy

dir(#path + date -> format(’yyyyMMdd')) -> create(480)

This will create the directory if it does not exist. With user permits of letting owner read, write and execute. Group can read. Others can’t do anything. Since this is done by Lasso the user and group will be the whatever the Lasso instance operates as (default _lasso and _lasso).

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Patrick Larkin-3
Where does it come from?  

I thought you meant an auto-incememted MySQL id.  


PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us

On Mar 2, 2015, at 10:43 AM, Jolle Carlestam <[hidden email]> wrote:

> 2 mar 2015 kl. 16:16 skrev Patrick Larkin <[hidden email]>:
>
>> How do you make that crazy GUID?
>
> It’s not crazy, it is standard.
> http://en.wikipedia.org/wiki/Globally_unique_identifier
>
> And in Lasso 9 it’s easy:
> lasso_uniqueid
> -> FBEBE980-FC1A-471B-B8B3-AB593FB65D5E
>
>>
>> And are you creating dated directories on demand with Lasso?
>
> Yes, again with Lasso 9 it is very easy
>
> dir(#path + date -> format(’yyyyMMdd')) -> create(480)
>
> This will create the directory if it does not exist. With user permits of letting owner read, write and execute. Group can read. Others can’t do anything. Since this is done by Lasso the user and group will be the whatever the Lasso instance operates as (default _lasso and _lasso).
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
2 mar 2015 kl. 16:44 skrev Patrick Larkin <[hidden email]>:

> Where does it come from?  
>
> I thought you meant an auto-incememted MySQL id.  

I never ever use Mysqls auto incremented ids in anything exposed outside the record itself. I store records using a GUID as unique identifier. On occasion I let Mysql create them for me. But usually I let Lasso do it.

local(id = lasso_uniqueid)

myDS_table -> add((:’id’ = #id, ’somefield’ = #somevalue))

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Patrick Larkin-3
Is there a GUID method in Lasso 8?

PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
——
Patrick Larkin
Application Management Group
Information Technology
Bethlehem Area School District
https://www.beth.k12.pa.us

On Mar 2, 2015, at 10:49 AM, Jolle Carlestam <[hidden email]> wrote:

> 2 mar 2015 kl. 16:44 skrev Patrick Larkin <[hidden email]>:
>
>> Where does it come from?  
>>
>> I thought you meant an auto-incememted MySQL id.  
>
> I never ever use Mysqls auto incremented ids in anything exposed outside the record itself. I store records using a GUID as unique identifier. On occasion I let Mysql create them for me. But usually I let Lasso do it.
>
> local(id = lasso_uniqueid)
>
> myDS_table -> add((:’id’ = #id, ’somefield’ = #somevalue))
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

decorior
Hi, Jolle:

What has always bothered me about this approach is that there is a finite chance you will generate the same id. How do you prevent that?

We have taken the approach of encoding the primary key when it is rendered so there is no relationship from the table to user since the key is encrypted with lasso.

That way we are guaranteed uniqueness, without exposing the id value.

Deco


> On Mar 2, 2015, at 8:51 AM, Patrick Larkin <[hidden email]> wrote:
>
> Is there a GUID method in Lasso 8?
>
> PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
> ——
> Patrick Larkin
> Application Management Group
> Information Technology
> Bethlehem Area School District
> https://www.beth.k12.pa.us
>
> On Mar 2, 2015, at 10:49 AM, Jolle Carlestam <[hidden email]> wrote:
>
>> 2 mar 2015 kl. 16:44 skrev Patrick Larkin <[hidden email]>:
>>
>>> Where does it come from?  
>>>
>>> I thought you meant an auto-incememted MySQL id.  
>>
>> I never ever use Mysqls auto incremented ids in anything exposed outside the record itself. I store records using a GUID as unique identifier. On occasion I let Mysql create them for me. But usually I let Lasso do it.
>>
>> local(id = lasso_uniqueid)
>>
>> myDS_table -> add((:’id’ = #id, ’somefield’ = #somevalue))
>>
>> HDB
>> Jolle
>>
>> #############################################################
>>
>> This message is sent to you because you are subscribed to
>> the mailing list Lasso [hidden email]
>> Official list archives available at http://www.lassotalk.com
>> To unsubscribe, E-mail to: <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
In reply to this post by Patrick Larkin-3
2 mar 2015 kl. 16:51 skrev Patrick Larkin <[hidden email]>:

> Is there a GUID method in Lasso 8?

I don’t know. I don’t do 8.

But, you probably already have a path you’re accustomed to using when it comes to storing records. No need to change that. You can still use a GUID to store the file name. Let Mysql hand it to you. Create a field in your file table and call it, for example fileguid. When creating the record, use an sql query to do it.
"INSERT INTO file SET
fileguid = UUID(),
filesuffix = ’” + #filesuffix -> encodesql + ”'
etc

You can then retrieve the GUID value by fetching the record you just created and use that to store the file.

If this seems too cumbersome then use the unique id that Lasso 8 can provide for you; lasso_uniqueid. It is far better than using anything predicable as when using auto incremented values.

Or write your own tag to supply a unique value.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Jolle Carlestam-2
In reply to this post by decorior
2 mar 2015 kl. 16:58 skrev deco rior <[hidden email]>:

> What has always bothered me about this approach is that there is a finite chance you will generate the same id. How do you prevent that?

As I understand it GUIDs are guaranteed to be unique. That’s part of their offer (Globally unique identifier).
Not only unique within the same context, for example a table. But unique universally. Enabling you to merge different tables still being ensured that the id value will be unique for all records involved. Try that with an auto incremented integer id.

I know that at Amtac Rick used a custom method to generate an id. If I recall it was 20 chars long. Part of that process involved checking that it wasn’t already used. Don’t think he ever got a collision. GUIDs are BTW 36 chars long.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

stevepiercy
In reply to this post by Patrick Larkin-3
On 3/2/15 at 8:43 AM, [hidden email] (Patrick Larkin) pronounced:

>Thanks for this.
>
>I’m guessing Lasso can still server a file even though Apache
>cannot “see” the directory?

Yes.

>Also, would just placing the directory completely outside then
>web root and configuring Apache to see it achieve the same thing?

Yes.

>I’m not sure how I’m going to name the files yet - it
>should be identifiable by a database record at the minimum but
>without a filename with an obvious naming convention.

knop_unique is "good enough" if you already use it.  There's
also several methods on TagSwap.  Search for "unique".  It's
possible that Johan released knop_unique as unique_id on TagSwap.
http://www.lassosoft.com/tagswap

>Once the directory is populated by 6000+ files, I’m hoping
>Lasso’s FILE tags are fast enough to parse through the
>directory and find the needed files upon request.  :)
>
>(Making a directory for each user is tricky since that number
>fluctuates constantly and is over 2000 users.  Active Directory
>could help this I suppose.)

There's lots of different strategies.  One issue to be aware of
is where you end up with 10s of thousands of directories or
files at any given level: if you ever do a directory listing of
the parent directory, sloooooooow.  However using the exact
filepath should not be a problem in this case.

Also consider using a CDN, which takes care of many of these
problems for you.
http://www.stevepiercy.com/articles/rackspace_cloud-lasso-sdk-for-the-rackspace-cloud-files-api/

--steve


>PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
>——
>Patrick Larkin
>Application Management Group
>Information Technology
>Bethlehem Area School District
>https://www.beth.k12.pa.us
>
>On Feb 27, 2015, at 3:31 PM, Steve Piercy - Website Builder
><[hidden email]> wrote:
>
>>Create a directory structure, something like this.
>>
>>/users/user1
>>...
>>/users/userN
>>
>>Configure Apache to deny access to the directory /users/.
>>
>>Store the username in the session data.
>>
>>Store a reference the filename in a database, and serve it up like this:
>>
>><a href="/filepath/download.lasso?file=my.pdf">my.pdf</a>
>>
>>In download.lasso, grab the username from the session data (else redirect to
>error.lasso because it is not an authenticated user), then do a file_serve:
>>
>>file_serve(include_raw($filepath), -file=$filename);
>>
>>You can also include the MIME type as an argument to file_serve.
>>
>>--steve
>>
>>
>>On 2/27/15 at 3:05 PM, [hidden email] (Patrick Larkin) pronounced:
>>
>>> Hello -
>>>   I’ve been tasked with building an application which
>>>permits our employees to upload
>PDF documents of certain clearances. By nature, these document
>could contain sensitive information.
>>> I have used Lasso 8 to build uploaders before that simply upload a document to a
>designated folder,  rename it to something, and add a database
>entry for the file’s existence.
>>> in this case, I cannot use a standard naming convention.  Even if I encrypted the
>file name to contain the particulars needed to identify the
>file, I’m afraid that people will try different combinations
>of characters to try and find one.  In other words, the
>directory containing the files will be exposed.
>>> Am I overly concerned?  Is there a way to secure this so that only the person
>submitting the files can “see” them (aside from the
>administrator or the system that needs rights to everything)?
>>>   Anyone do anything similar?
>>>    PLEASE NOTE:  My e-mail address has changed to:  [hidden email]
>>> ——
>>> Patrick Larkin
>>> Application Management Group
>>> Information Technology
>>> Bethlehem Area School District
>>> https://www.beth.k12.pa.us
>>>    #############################################################
>>>   This message is sent to you because you are subscribed to
>>> the mailing list Lasso [hidden email]
>>> Official list archives available at http://www.lassotalk.com
>>> To unsubscribe, E-mail to: <[hidden email]>
>>> Send administrative queries to  <[hidden email]>
>>
>>-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>Steve Piercy              Website Builder              Soquel, CA
>><[hidden email]>               <http://www.StevePiercy.com/>
>>
>>
>>#############################################################
>>
>>This message is sent to you because you are subscribed to
>>the mailing list Lasso [hidden email]
>>Official list archives available at http://www.lassotalk.com
>>To unsubscribe, E-mail to: <[hidden email]>
>>Send administrative queries to  <[hidden email]>
>
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Document storage and security

Bil Corry-3
In reply to this post by Jolle Carlestam-2
GUID do have a small risk of collisions, but "small" is actually really
really really really small.  Like "you'd be more likely to win the lottery"
small, assuming the GUID generation routine is cryptographically random.

Below is a solution I came up with many years ago that uses a random ID for
the row ID, checks if it already exists in the table, and if so, generates
another one until there isn't a collision.  That will guarantee no
collision.  You could substitute MySQL's UUID() function instead of my
custom one.

Speaking of MySQL's UUID(), it appears it has features to help prevent
collisions, so I don't think you need to worry about collisions if you're
using the MySQL function:

http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html#function_uuid


- Bil


-- Function "randomstring" DDL

CREATE DEFINER=`root`@`localhost` FUNCTION `randomstring`(i TINYINT
UNSIGNED) RETURNS varchar(255) CHARSET utf8
BEGIN

        DECLARE s VARCHAR(255) DEFAULT '';

        REPEAT

                SET i = i - 1;

                SET s = CONCAT(s,substring('1234567890qwertyuiopasdfghjklz
xcvbnmQWERTYUIOPASDFGHJKLZXCVBNM',FLOOR(1 + (RAND() * 62)),1));

        UNTIL i <= 0 END REPEAT;

        RETURN s;

END;

-- Table "mytable" DDL

CREATE TABLE `mytable` (
  `id` varchar(20) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
  `stuff` varchar(50) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

DELIMITER ;;
CREATE TRIGGER `randomID` BEFORE INSERT ON `mytable` FOR EACH ROW BEGIN
        DECLARE done TINYINT DEFAULT 1;
        DECLARE randomID char(20) BINARY DEFAULT '';
        REPEAT
                SET randomID = randomstring(20);
                SELECT count(*) INTO done FROM mytable WHERE mytable.id =
randomID;
        UNTIL done = 0 END REPEAT;
        SET NEW.id = randomID;
END;;
DELIMITER ;



On Mon, Mar 2, 2015 at 5:16 PM, Jolle Carlestam <[hidden email]> wrote:

> 2 mar 2015 kl. 16:58 skrev deco rior <[hidden email]>:
>
> > What has always bothered me about this approach is that there is a
> finite chance you will generate the same id. How do you prevent that?
>
> As I understand it GUIDs are guaranteed to be unique. That’s part of their
> offer (Globally unique identifier).
> Not only unique within the same context, for example a table. But unique
> universally. Enabling you to merge different tables still being ensured
> that the id value will be unique for all records involved. Try that with an
> auto incremented integer id.
>
> I know that at Amtac Rick used a custom method to generate an id. If I
> recall it was 20 chars long. Part of that process involved checking that it
> wasn’t already used. Don’t think he ever got a collision. GUIDs are BTW 36
> chars long.
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
12