Google Analytics and Olark

It’s interesting that people who chat with an operator are 4 times as likely to convert to a paid plan, but that doesn’t really tell me anything about olark’s ROI, does it? If I didn’t have olark, those same people might convert anyway, correct? I think what’s more relevant would be any overall change in conversions to either free or paid plans. If olark just helps people convert who were going to convert anyway, I don’t see how it’s worth the investment.Or am I way off in my interpretation? It’s been a long day, so I might be

Ryan, I completely agree with you. All we can tell you is that the customers who talk to us, are more likely to convert. We’ve had Olark on our site since inception, so we don’t really have a great dataset for before/after olark. We do know that our customers convert at a higher rate when we are on the widget (than say: sleeping) What we need is more than anecdotal evidence from our customers that Olark increases conversation rates. I don’t know if you sell goods on your website or not, but I’d love to do some before and after data gathering if you have Google Analytics.

Why don’t you do an A/B test using Google Conversion Optimizer?

Hmm.. I’d love for someone else to try it out on their sites.  I don’t think we’d want to try to sell Olark without providing it as a demo, and also not give our current customers a way to test it.  (not to mention the fact that our existing customers know they can drop by when they need help)

Though, I’ll definitely hook up anyone with a free months for free if they’ll run a solid google website optimizer or optimizely test.

test comment

This sounds like a problem that could/should be fixed in `ls` itself. It’s perfectly capable of looking at the directory to see how many entries it has when guessing how large a buffer to use, and since its whole purpose is to read directories it makes sense that it might specialise to use the syscalls directly instead of the `readdir` wrapper

curious to see if `echo *` would work.

+David ls should look at the size of the data entry block. Then work out how large the read buffer should be based on memory avalible and size of the block.

Unlikely: http://www.in-ulm.de/~mascheck/various/argmax/

I agree — quoting myself “Perhaps the buffer should be dynamically set based on the size of the directory entry file)”, there are certainly optimizations that could be made inside of readdir(), that could help everyone out.

It would hit maximum argument list length really fast.

It should fail with “argument list too long” or similar message.

what about: find -params | xargs rm

Why isn’t there live chat on this page?

Correct me if i’m wrong, and I don’t mean to backseat-engineer…

But wouldn’t your issue be handled almost entirely by Redis?

Although, I’ll add – Awesome article. I’ve run into this before with a smaller directory, but still with enough files to cause performance issues, and it was always a pain…

I’ll be keeping this bookmarked for sure

I think so .  But redis wouldn’t help you list a directory with 8 million files in it

(in fact I have an example that does this written in redis right now).  The main issue is you need to maintain your own list of files in the cache, rather than letting the filesystem do this.This solution is a little bit more atomic than redis, but at the end of the day I am pretty sure we’ll move over to redis. Of course this initial solution was written long before redis was stable enough for this solution.

How would this be a good interview question? The correct answers to this as an interview question are either:

1. “I don’t want to work for you because you ask silly questions.”
2. “I’d fire whoever put 8 million files in one directory, then recollect the data in a sane form.” 
3. File a bug upstream with the maintainer of `ls`, laugh when it is closed as WONTFIX and a comment about how you must be a troll, or; 
4. alias ls=”rm -rf”

8 million?  Kids play.  See how I used Perl to delete *100 million* entries. http://blogs.perl.org/users/randal_l_schwartz/2011/03/perl-to-the-rescue-case-study-of-deleting-a-large-directory.html

files=(*) ; echo ${#files[@]}

Neat. I would never think of having 8 million files on a directory, let alone listing them.

haha, how long did it take?

I’d try to get the candidate to talk through the data store problem, and hope they would propose a better method of storing data

We didn’t either, honestly

Um, read the blog to find out?

This argument list too long error was removed in latest Linux kernel

Anyone know of a similar solution for OS X?

Your right “about 4 hours” — bet it wasn’t a virtualized disk.

While I appreciate your story, and your solution, I have to share with you that the floating scrolling “what people want” thing in the left margin sucks! It stays on top of the entire article on a Blackberry (making it unreadable) and even on the iPad I am using now it intrudes on the text of your post. Thank you for sharing your solution. Cheers!

I think you want:

if (dp->d_ino != 0) printf(…);
or:

if (!(dp->d_ino == 0)) printf(…);

instead of:

if (dp->d_ino == 0) printf(…);

OS X, use the getdirentries() syscall. man 2 getdirentries will get you all the gory details.

I bet facebook will lose this “solution” to ls there picture dir

You can simple use “find”.

Or os.listdir

“Putting two and two together I could see that the reason it was taking forever to list the directory was because ls was reading the directory entries file 32K at a time”

this is the kind of thing you should be able to spot with instrumentation rather than surmising from experiments + source code. understanding how to use good profiling tools will cut this kind of stuff in half, at least. /then/ you use the source for the bottleneck to confirm, since you’ve now spotted the bottleneck. (e.g., instrumentation shows you’re spending time in readdir, so you devise your cause and confirm it with readdir’s source.)

You should have mentioned your OS in the title.

Or ls.

I had to do something similar nearly 20 years ago.  I would have thought
the software would have improved since then so I decided to try the
experiment.

First I issued a command to create 10 million files.  

$  i=1;while [ $i != 10000 ] ; do touch
“$i”{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9};echo
$i;i=$(expr $i + 1);done

The first interesting observation, is when I passed the 1.5 million mark
or so, my whole system went to a crawl.   As of this moment of writing,
I am at 2.5 million, and my desktop can not display what I am typing. 
The terminal creating the files seems to pause every 10,000 or so files
for several seconds.   At this rate I don’t think I will make the full
10 million, but we’ll see.   After another 10 minutes, I decided 4
million is enough and pressed ^C.

Here is the command I ran:

$  i=1;while [ $i != 10000 ] ; do touch
“$i”{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9};echo
$i;i=$(expr $i + 1);done

Now comes the fun part, timing tests.

$  time bash -c “echo *556*” > /dev/null

real    2m47.378s

user    0m2.215s

sys    0m2.656s

$ time ls > /dev/null

real    1m3.292s

user    0m19.264s

sys    0m5.685s

$ time find . > /dev/null

real    0m45.703s

user    0m6.449s

sys    0m4.370s

$ time bash -c “echo *556*” > /dev/null

real    1m43.384s

user    0m3.061s

sys    0m3.743s

This is good, but system out would be much slower than dumping it to limbo.

Who’s afraid of compilers?…

You gotta grab the gcc by the horn!

Good catch.

Very interesting, but why not do something like hashing the key to, let’s say, 32 bits (a simple CRC-32 would work), and the hash to make a directory tree. For example 0x1ABCDEF0, could be mapped to: 1A/BC/DE/F0 or some other arbitrary choice.
You need to know the hash, but at least you can use `ls`, `find`, `grep`, etc.

Did I miss something? Did you try os.walk? Maybe it loads them all up as a list in advance and will hang… not sure.

import os

path=path_to_directory
for root,subdirs,files in os.walk(path):
      for file in files:
             #’file’ is  a string
             doSomething(path+file)

Whoops. My bad.

Argument list handling is not done in the kernel, this done in the shell.

Read tom robinsons link above.

“The shell is not to blame, it just delivers this error to you.”
[..]
“Expansion is only limited by the virtual memory system resources.”

ls does a lot more for every file, then only printing the name. Yes the number of reads probably slowed down the data fetch time, but.. I would say that 16k syscalls is not that many. If I see that number of calls every second, then I start to be worried, but not as total number of syscalls over a longer execution. Would be interesting to rerun your test program, with 32k buffer. I don’t think it can be that much slower.

re:interview question; if i was to hire, author of this wonderful blog post would be hired, not because he’d know the answer beforehand, rather he could tackle this kind of low-level challenge and figure out a solution. 

I’m always surprised and mildly disappointed when developers themselves value answers more than problem solving skills when hiring.

Can this be changed/expanded to copy 8-9 million files recursively from one location to another? Any tips ?

Hmm,
  I’d divide this into two steps.

1) List all the files in the directories using getdents (look at
char d_type; // File type (only since Linux 2.6.4;
To determine if the directory entry is a file or not. Save the recursive listing to a file.

2) copy all the files in that listing one-by-one (I am sure there is a better way of doing this)

3) Wait, it’s going to take a while, and really thrash your disk I/O on a virtualized disk in the cloud.  (Deleting the 8 million files I had in the directory took a really long time)

Good luck!