Forum Moderators: open

Message Too Old, No Replies

Google Chrome can't handle REALLY long JavaScript string concatenation

         

KenB

6:01 pm on Jan 26, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My site uses a JavaScript based cascading top menu, which works great in all browsers except for Google Chrome. The JS that generates the top menu is a collection of variables and functions that contain pieces of the final HTML that will get written to the browser. For instance, here are a few small snips to demonstrate the gist of what I do:

var AH='<A HREF="http://',S=' class="TMLI"',X='<li'+S+'>',R='<li',Y='</li><li',L=Y+S+'>',Z='</li></ul>';
function mU(L,i){return'<ul class="TMUL TML'+String(L)+'" id="m'+String(i)+'">'}
function mJ(a,c){return' onmouseover="ieH(\'m'+String(a)+'\');" onmouseout="ieO(\'m'+String(a)+'\');" class="TMLI TMA'+String(c)+'">'}

The idea is that by replacing commonly repeated pieces of HTML in the top menu with variables and functions I can reduce the file size of the JavaScript tremendously. The menu contains around 300 links worth of nested UL lists would be about almost 90kb of HTML, but is around 30kb of JavaScript which compacts down to about 12kb when gzip is applied. Since the menu is stored in an external JS file, the code is pulled from my server once and is then reused on all pages as users request other pages. What this means is that a highly functional site directory is readily available to users without incurring serious download requirements (yes there are alternative navigation methods for those without JavaScript & bots).

I've used this menu for years and it works great in IE(6,7&8), Firefox, Safari and Opera. The problem is it doesn't work in Google Chrome.

After a tremendous amount of trial and error I finally figured out that If I cut the menu down in size it would also work correctly in Google Chrome. It didn't matter what part of the code I took out, what seemed to matter is that the master variable holding the final concatenated string got below a certain length. Basically Chrome is unable to handle variables that are longer than a certain threshold.

My end solution was to break the menu into two master variables and then concatenate the two master variables together just as I returned the results. For example:


return M1+M2;

Google chrome seems to be happy with this.

Just some food for thought if you do some massive string work and Chrome doesn't like what you are doing, break long strings into a couple of smaller variables, it might solve your problems.

Fotiman

6:50 pm on Jan 26, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



A couple of comments.

1. How long is "long"?

2. That sounds like a maintenance nightmare. Cramming all of that into JavaScript in an effort to save 60kb? I don't know if I would have found much value in that effort. Also, what sort of performance hit are you taking by running this through the JavaScript engine first? You might be saving a tiny amount of space on one end, but when you add the processing time (and extra memory required to store the HTML in a JavaScript variable), plus the utter complexity of trying to modify it and keep it in sync with whatever non-JavaScript solution you're using... is it really worth it?

I'm not trying to discourage you, I'm just wondering if that approach is a good long term strategy.

In any case, have you used any of the Chrome developer tools to try and debug what is happening?

KenB

8:16 pm on Jan 26, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If the entire menu was housed in a single variable in memory, after concatenation it would be at least 85kb.

It isn't a maintenance nightmare because the whole thing is generated via PHP on the fly. I have a master text CSV file that holds the linking instructions for the main menu items. This master file generates all the menus on my site not just the JS menu. I update that one CSV file and everything is updated.

The PHP code also automatically pulls in my RSS feed from my blog which is on a sub-domain and separate server so that the latest blog entries get automatically added to the menu. I use a caching mechanism to cache my blog RSS feed on the main server to eliminate the latency of making a request to a different server.

The same master JS file gets used by my main site, sub-domains and partner branded sites so that they all always have the same top menu and all I have to do is update a single CSV file to pull it off.

This master JS file also does more than generate the top menu, it handles my other common JS functions as well. All told the transfer requirement (headers etc.) for that file is just over 13kb in size gzipped. When I had the top menu JS code separate from my other JS code the combined download requirement was about 17kb so it is really nothing as far as download goes.

This master JS file is set to be cached by the browser and cache engines for 24 hours, which is longer than most of user sessions, but is short enough to make sure the top menu doesn't get stale.

All told based on Opera and Chrome's developer tools network speed notes, it takes me around 200ms for those browsers to request and then receive the JS file in question. As a comparison a typical 1kb image from my server takes about me 80ms from request to delivery. I also usually have a ping latency to my server of around 40ms.

To download the main HTML, the CSS and master JS file then render past the main menu takes less than one second according to both Chrome and Opera. IE7 is also pretty good with rendering to this point.

The reason the menu doesn't slow down rendering is that only the parent menu bar needs to get rendered. The drop down menus don't render until the mouse hovers over the parent menu item.

The biggest lag I see in rendering happens when the ad code scripts start to fire but I've structured the layout of my pages such that all of this fires after the main content is rendered. As such users have full access to content about 1/3 of the way through a page loading.

What would really help is if my ad providers cleaned up their HTML, CSS and JS as well as used shorter cookie strings. I could probably knock off over 1sec of download/rendering if their code was minified.

Is the code worth it? Yes I think it is. Here's why:

1) Using a separate JS file allows me maintain a single unified menuing system for users across multiple "sites", which provide for consistent navigation.

2) A separate CSV file and automatically pulling RSS feeds keeps maintenance very simple.

3) By using a separate JS file I'm able to update the top menu on the partner branded sections of my site, which are housed on partner servers, without having to submit a template change request.

4) Having the menu in a separate file allows the menu to be cached for the duration of a session thus eliminating a significant amount of page bloat, which allows pages to render extremely fast on multiple page visits by users.

5) By having a robust menuing system at the top of every page I'm able to get users to visit more pages on my sites. The more interesting stuff users find on my site, the more likely it is that they will return and tell others about my site.

6) By having the menu in a JS file that is blocked to robots by my robots.txt file the menu doesn't create the appearance of a link flood on my site. These links after all are intended for users not bots.

KenB

10:18 pm on Jan 26, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



1. How long is "long"?

I just figured out how to determine how long it took Google Chrome to process the script in question. T

On my computer, which is old and pretty slow (WinXP Pro SP3 running on a 1.8gHz Intel Pentium M processor and 1gb of RAM), the top menu function took 3ms (3/1000 of a second) to parse. So I'd say that scripting wise it has zero impact on page rendering.

Seb7

11:23 pm on Jan 26, 2010 (gmt 0)

10+ Year Member



Years ago strings had a max limit of 64kb but then it moved to 2gb. I wonder if Chrome is limited to 64kb? Which might explain its faster performance.

KenB

11:52 pm on Jan 26, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



64kb could easily be the max string size. It sounds close to the amount of menu I chopped out when things started to work right in Chrome.

To be clear if my full menu was turned into a string in a single variable before being returned it would fail. When it is concatenated into two different master variables and then those two variables are concatenated together on the return command it works just fine.

This would fail:


var m1="1st half of menu";
var m2="2nd half of menu";
var a=m1+m2;
return a;

This would work:


var m1="1st half of menu";
var m2="2nd half of menu";
return m1+m2;

It took me quite a bit of testing and digging to discover this. I wasn't getting any errors and I just couldn't find any bad code. I started commenting out sections of the PHP code that generated the menu to try and isolate out the problematic code. After isolating out what I thought was the bad code and not finding anything. I decided to enable that section of code and disable a different section to see what happened. This is when I discovered the code wasn't bad but that Chrome was choking on the master variable's string size.

My hope is that by starting this thread I'll save someone else some major frustration. I know that Webmaster World threads showing up in Google SERPs have saved me considerable frustration on more than one occasion.

Fotiman

7:19 pm on Jan 27, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The limit is definitely greater than 64k. Easily tested by doing this:


<html>
<head>
<title>Variable limit test</title>
</head>
<body>
<div id='N'></div>
<script type="text/javascript">
var ONE, TEN, HUNDRED, THOUSAND, TENTHOUSAND;
function timesTen(v) {
var i, s = '';
for (i = 0; i < 10; i++) {
s += v;
}
return s;
}
ONE = 'A';
TEN = timesTen(ONE);
HUNDRED = timesTen(TEN);
THOUSAND = timesTen(HUNDRED);
TENTHOUSAND = timesTen(THOUSAND);
document.write("<li>ONE.length = " + ONE.length + "<\/li>");
document.write("<li>TEN.length = " + TEN.length + "<\/li>");
document.write("<li>HUNDRED.length = " + HUNDRED.length + "<\/li>");
document.write("<li>THOUSAND.length = " + THOUSAND.length + "<\/li>");
document.write("<li>TENTHOUSAND.length = " + TENTHOUSAND.length + "<\/li>");
var i, str = '';
for (i = 0; i < 100000; i++) {
str += TENTHOUSAND;
}
document.write("<li>str.length = " + str.length + "<\/li>");
</script>
</body>
</html>

This easily creates a 1,000,000,000b string.

KenB

7:45 pm on Jan 27, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What happens if you try to create a massive string by concatenating in the following fashion:


var a='bla'+b+c+'bla bla'+e+jF(1000)+e+'keep connecting item after item with a plus sign instead of doing str+= on different lines.';

Think along the lines of a single variable assignment via contamination that is around 35kb in size. I'm not talking the length of the resulting string that gets written, but the length of the concatenation instruction after the equals sign and before the semicolon.

Literally to fix things and make them work I took something like this:

var m=a+b+c+'a whole bunch of text'+c+d+sF(100)+'a lot more text'+a+z+'a whole bunch more text'+q+w+x+y+c+'yada yada yada';
return m;

and broke it up into two variables by replacing one plus sign with a coma new variable name and an equals sign like this:

var m1=a+b+c+'a whole bunch of text'+c+d+sF(100)+'a lot more text'+a+z+'a whole bunch more text'+q,m2=w+x+y+c+'yada yada yada';
return m1+m2;

That's it, I simply broke the instruction up into two parts and made no other changes and the darn thing worked beautifully. When I tried to concatenate the two variables into a single variable before sending it to return, the whole thing stopped working again. For instance the following would break:

var m1=a+b+c+'a whole bunch of text'+c+d+sF(100)+'a lot more text'+a+z+'a whole bunch more text'+q,m2=w+x+y+c+'yada yada yada',m=m1+m2;
return m;