SiteSetting class need redesign

8/25/2006 10:49:10 AM

kwa

Total Posts 23

View Posts

Re: SiteSetting class need redesign

I just discovered the error in the new page I created was because of SiteSetting is null. That is because SiteSetting was initialized in the Global.asax.cs then stored in the context. What the method SiteSetting.GetCurrent() does is nothing more than retrieve object from the context. My suggestion is let the method GetCurrent() responsible for the existence of the object, it take care everything including caching and storing object in context. This is where the caching implimentation takes place.

In the global page, instead of creating SiteSetting object, just call the GetCurrent().

8/25/2006 1:34:38 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Hi Bo,

I think you should subscribe to the svn commit notification so you can know about changes committed to svn
http://forge.novell.com/mailman/listinfo/mojoportal-svncommits

I've made some changes recently in relation to SiteSettings in svn /branches/joesandbox2

However I still think SiteSettings should be created in Application_BeginRequest and added the http context this way it is available to all users controls throughout the request duration.

I added the SiteSettings.GetCurrent a while back becuase I found myself writing the same code in many places
if(Context.Items["SiteSettings"] != null)
{
siteSettings = (SiteSettings) Context.Items["SiteSettings"];
}

It doesn't seem unreasonable to me that SiteSettings.GetCurrent can return null
If it returns null in your page your should try and find out what happened and why it did not get added to the context at the beginning of the request.

I think we should first look into implementing page caching before we look at caching SiteSettings

8/25/2006 2:25:17 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Well, after thinking about it more. It might be better to not add SiteSettings to the context until the first time its needed.

I think in order to to that I will need to move SiteUtils from the Web project into the business project because we need some of the methods from there to get the host name and other things and we don't want a reference from the business project to the web project. I've been thinking this might be a good idea to move SiteUtils out of the web anyway so that external proejcts can also access SiteUtils and its not just avilable in the web project.

And probably I will remove SiteSettings.GetCurrent and create SiteUtils.GetCurrentSiteSettings because it would be better if we don't have dependencies on System.Web in a lot of business classes and only have it in SiteUtils. This way the business classes don't need to know about httpcontext and other System.Web things

8/25/2006 10:33:06 PM

kwa

Total Posts 23

View Posts

Re: SiteSetting class need redesign

In the method GetCurrent()

if(HttpContext.Current.Items["SiteSettings"] != null)

{

return (SiteSettings)HttpContext.Current.Items["SiteSetting"];

}

If HttpContext.Current.Items["SiteSettings"] is null , then do nothing. ----> that is why SiteSetting was null. The GetCurrent() method does not guarantee an instance of the object.

If you want the SiteSetting available to all controls, I came up with a new way. SiteSetting should not be created based on pageId and pageIndex. The new scenario is to pull all data and store it in the cache. Create some methods to get setting for each page. Whenever the method GetCurrent() get called it try to get sitesetting from the cache. If no cache, get it from database, store it in the cache and finally return instance of SiteSetting. This way, it is guaranteed that object is never null.

public static GetCurrent()

{

SiteSetting setting = (SiteSettings)Context.Cache["SiteSettings"];

//check if it is null which mean no cache

if(setting == null)

{

//repopulate settings

setting = new SiteSetting();

//cache it

Context.Cache.Insert("SiteSettings",setting,null, DateTime.Now.AddMinutes(60),System.Web.Caching.Cache.NoSlingExpiration);

}

return setting;

}

Modifying GetCurrent() required a new way of creating new instance of SiteSetting too. SiteSetting should be able to get the Hostname and fetch all data related to this Hostname.

Some additional methods are also needed such as method to get active page, method to get setting for a specific page ID etc.

That is why I'm thinking I need a serious modication to SiteSettings class. If you want, after I am done, I can send you a copy of what I do so you can see it.

(Bytheway, I also came up with a new and better way of caching that guarantee performance and flexibility, admin and user will see difference page cached, I will talk with you about this when I get the SiteSettings fixed).

8/26/2006 4:01:14 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Hi Bo,

I'll be glad to take a look when you are finished.
But, again I would urge you to track svn because I made a lot of changes last night.
SiteUtils is now in the business project
SiteSettings.CetGurrent no longer exists
SiteUtils.GetCurrentSiteSettings checks if siteSettings is in the context and if so returns it, otherwise it creates it adds it to the context and returns it

I am more interested in a page caching solution first before considering caching of siteSettings

Page caching must take into account:
the query string
the user roles
the user browser language
maybe other things I haven't thought of that testing will reveal
and must not cause excessive memory use, when load testing we must vary these things enough to see what will happen to memory in real environments

8/26/2006 3:46:13 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I have to admit you've made me think about SiteSettings and I see some good opportunities for refactoring. The changes to use the 2.0 .NET SiteMap make the current implementation less than ideal where it was a pretty good design for 1.1 .NET

Since the SiteMap is already cached it is unneccesary to retrieve the PageSettings collection on every request so I'm thinking (and this may be similar to what you are thinking), that the SiteSettings can be decoupled to some extent from the PageSettings collection (menu) and both SiteSettings and the PageSettings collection can be cached and only the ActivePage needs to be maintained in the HttpContext and probably won't need to be cached if we implement page caching

Thanks for making me review this implementation. I will make these changes and probably commit to svn tomorrow.

Cheers,

Joe

8/26/2006 7:47:46 PM

kwa

Total Posts 23

View Posts

Re: SiteSetting class need redesign

Yeah, now we are talking. ActivePage does not need to be cached or stored in the Context. SiteSettings encapsulates everything. SiteSettins works like a database in memory. Whenever the activepage is needed, just pull it from the collections. That will be faster. That is what in my to-do list. Because it is not a fast jobs, I did not get to do it; instead I just made a quick change to GetCurrent() and added another helper class. Here is what I did:

using System;
using System.Collections.Generic;
using System.Text;
using System.Web;

namespace mojoPortal.Business
{
    public class SiteSettingsHelper
    {
        #region Added by Bo on 8/26/2006
        public static SiteSettings GetCurrent()
        {
            HttpContext context = HttpContext.Current;
            //try to get it from cache
            SiteSettings setting = (SiteSettings)context.Cache[GetCacheKey];
            //no cache ?
            if (setting == null)
            {
                //create new object
                setting = new SiteSettings(Hostname, PageIndex, PageID);
                //insert into cache
                context.Cache.Insert(GetCacheKey, setting, null, DateTime.Now.AddMinutes(3), System.Web.Caching.Cache.NoSlidingExpiration);
            }
            return setting;
        }
        private static string Hostname
        {
            get
            {
                HttpContext context = HttpContext.Current;
                return context.Request.ServerVariables["SERVER_NAME"].ToLower();
            }
        }
        private static int PageID
        {
            get
            {
                HttpContext context = HttpContext.Current;
                int pageid = 0;
                if (context.Request.QueryString["pageid"] != null && context.Request.QueryString["pageid"] != "")
                {
                    pageid = Convert.ToInt32(context.Request.QueryString["pageid"]);
                }
                return pageid;
            }
        }
        private static int PageIndex
        {
            get
            {
                HttpContext context = HttpContext.Current;
                int pageindex = 0;
                if (context.Request.QueryString["pageindex"] != null && context.Request.QueryString["pageindex"] != "")
                {
                    pageindex = Convert.ToInt32(context.Request.QueryString["pageindex"]);
                }
                return pageindex;
            }
        }
        private static string GetCacheKey
        {
            get
            {
                string key = "SiteSettings";
                if (PageID > 0)
                {
                    key = Hostname + "-SiteSettings-" + PageID;
                }
                else
                {
                    //PageID = 0 or no pageid
                    key = Hostname + "-SiteSettings-default";
                }
                return key;
            }
        }
        #endregion
    }
}

In the SiteSettings class I modified the GetCurrent() method to:

public static SiteSettings GetCurrent()

{

return SiteSettingsHelper.GetCurrent();

}

Well, the goal of SiteSettingsHelper is to cache different versions of SiteSettings based on PageID and Hostname. And to not modify the structure of the portal.

This is not what I really planned to do and it is not the best way, but it is the fast and easy way to make things work and also a quick implementation of caching. Plus, I save alot of time.

8/27/2006 3:22:01 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I think it will be better to de-couple PageSettingsCollection and ActivePage from SiteSettings we don't need an in memory database and we don't need a different copy of siteSettings per page.

If I de-couple them then we only need 1 copy of siteSettings per site in the cache
We don't need to do anything about caching the PageSettings collection for the menu because the SiteMap is already cached, so no need for this chunk of data to live inside siteSettings at all just need to make SiteMapProvider get its data directly from the PageSettings class with a new method and once it has it SiteMap is cached and it won't call this method again until the cache expires

Getting the Active page is not just a matter of finding it in the PageSettings collection using pageindex, currently inside siteSettings we are loading all the modules for the active page but not for the other pages and I don't want to change to load the modules for all pages. My plan is to move this outside of siteSettings so that we only need one instance of siteSettings in the cache per site. I will add a new method SiteUtils.GetCurrentPage which will check the context for the CurrentPage object and if its not there create it and add it to the context then return it. CurrentPage will be equivalent to what siteSettings.ActivePage was and will have the modules collection for the current page.

PageIndex is a legacy thing that is only used by the skmMenu that we were using before 2.0 .NET. Having pageindex=somenumber in the query string was only so skmMenu whould know how to highlight the current page in the menu
PageID in the query string does indicate the active page id

We could cache a PageSettings instance per page (aka Current/Active page) but I think that caching the page itself will be better and make this un-needed so for now I'm going to continue putting CurrentPage/ActivePage in the httpcontext

I will commit these changes later today.

8/27/2006 4:48:43 AM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

We could cache a PageSettings instance per page (aka Current/Active page) but I think that caching the page itself will be better and make this un-needed so for now I'm going to continue putting CurrentPage/ActivePage in the httpcontext

If PageSettings contains everything needed to render the page, I recommend measuring the difference between caching the PageSettings and caching the rendered page. Caching the PageSettings would be a lot less effort than caching the rendered page because you wouldn't have to deal with the security and language issues. Also since caching the PageSettings would require less memory it might actually turn out to provide better performance due to locality of reference (e.g. the info needed to render the home page might fit entirely in the processor's cache, but the different versions of the rendered page might not).

Bottom line: "Measure twice, cut once."

--Dean

P.S. The one place where caching the rendered page would definitely provide at least some benefit is if you let the browser cache the page. That would obviously reduce network bandwidth. I'd recommend considering that as a separate optimization, which could be considered in addition to either caching the PageSettings or caching the rendered page.

8/27/2006 4:57:30 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Good point Dean.

I'll definitely measure the performance gain from caching pageSettings per page after I get siteSettings cached and decoupled and before doing anything with page output caching

PageSettings contains a list of modules for the page. Rendering still requires loading those modules which in turn call the db for their content so my guess is page output cache would offer much better performance though as you note it will be more difficult to implement

Good idea on trying to let the browser cache as well. If I do that is there a way to make the browser reload the page when the content changes or if the user logs in and has edit permission?

8/27/2006 5:45:22 AM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

I agree that output caching will probably provide better performance if caching PageSettings doesn't eliminate all the DB calls. An intermediate option would be caching in the business object layer so that you wouldn't need to load the modules and content from the DB each time. That would allow you to avoid the security issues but you'd still need to implement logic to remove objects from the cache when they are updated. You'll need that logic to do output caching anyway so it wouldn't be wasted effort.

Proposed plan:

Measure current performance.
Measure performance with "dumb" output caching (i.e. neglecting security and cache invalidation issues). This would give you an upper-bound on the effect of output caching.
Measure performance of "dumb" business object caching (i.e. neglecting cache invalidation issues). This would give you an upper-bound on the effect of business object caching.
Measure performance of both types of caching using simplest possible cache invalidation logic (ie. any change to anything on the site invalidates the entire cache.).
If it is worth the extra effort, implement more sophisticated invalidation logic and/or handle the security issues associated with output caching.
If it is worth the extra effort, implement browser caching. If you handled the output caching security issues in step 5, this is a no-brainer. If not, you need to decide whether the reduced network bandwidth and server load is worth the effort to do that.

is there a way to make the browser reload the page when the content changes or if the user logs in and has edit permission?

I believe so. I think there is a property you can set to force the browser to always revalidate it's cache entry with a HEAD request. So, you'd just need to implement Global.GetVaryByCustomString() to generate a different strings for users with different permissions and for changed content.

--Dean

8/27/2006 1:32:03 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Alright I've refactored SiteSettings moving the pagecollection and the current page out of its internals so it could be cached.
I've also done a little minor tweaking to the Cache feature that is already built into SiteModule.

I ran a baseline test this morning before making any changes to SiteSettings

Baseline Before Site Settings refactoring

After Refactoring SiteSettings

Then I tried setting a cache time of 360 seconds in the module settings of each module on the test pages.
The next test I ran was actually a little slower so I reviewed the CachedSiteModuleControl.cs and noticed it was using Context.Cache rather than Httpruntime.Cache, wasn't sure if changing that would help but it apparently did. I also made a few other tweaks so that the cache key would be different for users that can edit vs thos that can't and by current culture.

After Applying Tweaked SiteModule cache

Pretty good improvements!

8/27/2006 1:50:59 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

And here are the results using "dumb output caching" like this:
<%@ OutputCache Duration="120" VaryByParam="*" %>

with no handling for roles, culture etc.

8/27/2006 5:06:19 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Nice! A couple questions:

To ensure that the test machine wasn't the bottleneck, can you confirm that ACT was running on a machine other than the server and that the ACT machine's CPU was substantially less than 100% during the tests?

To ensure that the network wasn't the bottleneck, can you confirm that the total bandwidth consumed during the test (i.e. average response size times requests/sec) was substantially less than the bandwith of the connection between the ACT machine and the server?

Was the server CPU usage at roughly 100% during all the tests? If not, a more realistic test would probably use a larger number of simultaneous browser connections. By only using one browser connection I think you might be forcing the requests to occur sequentially. If the server CPU is idle waiting for a DB call associated with the only outstanding request, you will see less requests/sec than you would expect in a the real world where the server handles requests from multiple browsers on different threads.

--Dean

8/27/2006 5:33:32 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

ACT was running on the same machine as the web. I'm not positive whether the machine was pegged but I was watching and asp.net was using 50-60% of the processor and memory use was between 45 and 60MB. Memory probably can be reduced but not horrible I think.
machine is my laptop, a Dell Precision M70, 2.13GHz Pentium M with 2Gigs of RAM

Also was using debug compilation and debug build, results should be better on a remote machine in release mode.
I have another machine I could test on (and probably will soon) but I figured since all the tests were run in the same environment, relative results would still be meaningful.

My test was just using the ACT defaults and ACT recorded a browser session to create the test, just an anonymous session clicking all the pages in the menu. The output is all VB Script so probably very extenable, just haven't learned all the bells and whistles of ACT yet so started out with something basic

8/27/2006 5:51:17 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Its going to be challenging to come up with a strategy that really works for the Global.GetVaryByCustomString
The problem is a user may have page edit permission in which case they can edit any module on the page regardless of module specific permission. But a user may also have just edit permission on a module and not all modules on a page so showing the correct edit links for authenticated users could get very complicated.

Still, if its possible to gain 82 requests/sec, that is substantial

Short of that the current caching of siteModule is probably better than caching business objects since it is caching the actual rendered content.

I don't know yet if ACT can show us the benefit of browser caching. I'll look into it.

Joe

8/29/2006 5:10:19 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

I agree that writing an effective GetVaryByCustomString() is not going to be easy. For my proposal, see the first 3 bullets of my 8/18/2006 12:12:04 AM post on the "Improve performance 20%" thread. To handle page-specific permissions in addition to module-specific permissions, just treat the page like another module for the purposes of my proposal.

--Dean

8/30/2006 2:30:06 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I did some thinking about this last night and it just seemed like it would get too complicated to make sure the right cache copy gets shown to each user without having an explosion of cached copies.
Your suggestions seemed on the right track but thinking through how to implement them my train of thought couldn't quite pull the load ;-). I get the feeling I could spend a lot of time on it and it may not really improve performance as much as the dumb outputcaching does. I think calculating the vary string would not be too bad at the time when the page is being rendered from the db but once its in the cache the logic of having to calculate it correctly based on the user's role cookie, the requested path and all these variables in memory makes my brain hurt.

I think for now I'm going to punt on the page output cache strategy and work on other things, though I'd certainly welcome if anyone else wants to explore implementation of this.

I think the gains we have now are pretty good. I'm going to do some more testing using a remote machine and try to find a few other low hanging fruit performance improvements and then maybe try and ship a minor release before I get back to working on the SiteOffice features.

8/30/2006 2:53:55 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Sounds like a good plan. I agree that the implementation would be quite hairy and it's not at all clear it would be worth the effort.

8/30/2006 3:47:42 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

It just occurred to me that the way to track the variables needed would be to go the other way from what I just did and add more things to SiteSettings and put it in the cache.

We would need the PageSettings for all the pages within the site (which I just removed ;-)) but in addition to the ModuleCollection for the ActivePage which we had before we would need it for every PageSettings in the collection, this would enable us to do what Bo (kwa) suggested earlier in this thread and just choose the active page from the collection. This would basically be a dictionary of the site pages and modules with roles and with this and the user cookies we would have everything we would need to calculate some kind of string for GetVaryByCustomString()

We could set a timestamp or create a guid string on the SiteSettings object when it is first instantiated and use it as part of the vary string too. This way any dependency that causes the siteSettings cache to be refreshed would start creating a new set of pages.

The formula for the string might be something like
HasPageEditPermission.ToString()
If the user has page edit permision we calculate true for each module in sequence and add it to the string
else we call Module.HasEditPermision in sequence and add that to the string

What do you think Dean? Any flys in the ointment?

Would it be too much to store in memory for very large sites? Other problems? or do you think its worth experimenting with it? Might make initial loads slow when the application starts.

8/30/2006 4:54:13 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Sounds about right to me. We should probably also provide a way for modules to indicate cache expiration time. The expiration time for the page should be computed as the earliest expiration time of any module on the page. As an example consider a blog module which displays a calendar. Strictly speaking, a page containing that module should not be cached past midnight.

On a related note, you might want to look into techniques to reduce the overall size of pages. The mojoportal.com homepage currently weighs in at about 100Kbytes (not counting external files like images, styles, etc). Since your earlier benchmarks seem to imply that about 80% of time is spent just sending bytes to the browser, anything we could do to reduce page sizes could have a large impact on performance. For comparison if I convert the HTML to plain text, the resulting size is only 18Kbytes. So, roughly speaking, we are sending 100Kbytes of HTML to display 18Kbytes of text. Looking at the HTML source the 2 things that jumped out at me were 1) ViewState accounts for about 7700 bytes, and 2) the HTML for the calendar accounts for over 15KBytes. The calendar size could be reduced alot by eliminating the inline styles. Inline styles are taking up a lot of space elsewhere on the page too. It wouldn't suprise me if you could get a 20% performance increase just by trimming some of these kinds of fat.

8/30/2006 5:13:29 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Excellent idea on trying to trim the fat on the page size! The calendar is one of the few controls using the theme.skin file style properties and I think this is what makes inline styles where instead of specifiying things like border fonts and colors in the theme.skin, if we only use the skin to set css classes then it won't render so much inline style. Imagine how big the page was before we started using the CSS adapter for the menu it was rendering as a mess of nested tables with inline styles.

I'm thinking the SiteSettings class could store some key in the cache for each module and we can use a file dependency for each module with a callback method in siteSettings so after an update we touch the file and that raises the callback in siteSettings which can then re-load the module.

I still worry about that initial load thing. Sites that don't get much traffic are the worst affected by that because a higher percent of visitors see the first request which is slow under normal conditions due to the jitting but then this loading of the dictionary on a large site could make that request seem pretty slow. I've seen his before with other .NET sites, once I wrote a windows service called SitePoker that would request a page from sites listed in an xml file site every 15 minutes just to keep them alive, you know just give it a poke every few minutes ;-)

8/30/2006 5:32:09 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

The "after an update we touch the file and that raises the callback" scheme is good but it won't address content that just goes stale at some point in time even if there is no update (e.g. the day that is highlighted on the blog calendar). Also, touching a file would work fine in a single server environment but might not work in a web farm. There are a couple ways to address that, but I thought it was worth mentioning...

Initial load is definitely a concern. Couldn't you load the necessary information on demand instead of loading it all in response to the first request? I think I'm basically just suggesting that you cache the business objects...

8/30/2006 5:49:04 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Hmm, I hadn't thought about the web farm issue with file dependencies, I'd be interested in your suggestion for a web farm friendly approach.

I agree about the timing of the calendar I guess it just matters how long we cache the page in outpute cache. if we cache it for up to 5 minutes or so its probably trivial but if we're talking aching for hours then yeah we need to figure out something for midnight expiration.

Good idea loading on demand, we could just load modules a page at a time as pages are requested that would help a lot with the initial load issue. So we don' populate the whole site dictionary upfront, we add to it as needed, this also address my concern about how big the siteSettings obejct could get to some extent.

I saved the home page of this site a few minutes ago and it was 120kb. i was able to trim 20kb just by taking a few things out of the theme.skin file.

I'm about to crash but I'll work on this some more tomorrow night.

8/31/2006 4:55:53 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I setup my remote server for testing tonight, its on my local network
PIII 800Mhz 1GB RAM
Win2003 Server running IIS and MS SQL 2005

I used a hostname to create my test using http://loadtest/mojo and for a baseline I pointed it first at my local machine, laptop specs mentioned in previous post, identical db on both machines
While running the tests all on the same machine the processor was pegged with aspnet using 50-60% cpu on avg
While running the tests agains the remote machine, the ACT machine had about 80% system idle and the remote machine was pegged at 100%cpu with w3p using 80% on avg

Apparently the hardware is sufficiently better on the local machine that results were much better there

Everything on local machine

hitting the remote PIII 800 machine

I still think the remote machine is probably better for testing but it just goes to show we really can't get absolute numbers as its very hardware dependent. The main thing is to get a baseline and keep tabs on how performance changes as changes to the code are made and as new features are added over time.

I haven't yet started work on the new fangled siteSettings experiment or the page size reduction campaign. Probably this weekend, but at least I'm setup for easy testing.

8/31/2006 5:00:24 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

For web farms, I suggest story the last modified time (for each site, page, and/or module) in the DB. GetVaryByCustomString() should return a value that includes the relevant last modified time(s) retrieved from the DB. That means doing a DB call for each request, but I don't think that will be a big performance hit. If it is, using a temp table might help.

The issue with the calendar was just an example. The larger point is that a module's output might change due to circumstances other than user action. Assuming you want to allow add-on modules to support caching, I think we need a more general approach. At the moment, I'm thinking that SiteModuleControl should have an additional method called GetVaryByCustomString(). Global.GetVaryByCustomString() would instantiate and configure the SiteModuleControls associated with the requested page and would then call each SiteModuleControl's GetVaryByCustomString() method, appending the results. To handle the blog calendar case, BlogModule.GetVaryByCustomString() would return a string that contained, among other things, the current date. That would force a new page to be rendered each day.

Make sense?

8/31/2006 5:23:52 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

<
For web farms, I suggest story the last modified time (for each site, page, and/or module) in the DB.
GetVaryByCustomString() should return a value that includes the relevant last modified time(s) retrieved from the DB.
That means doing a DB call for each request, but I don't think that will be a big performance hit.
If it is, using a temp table might help.
<

Something like this does seem to be the only solution to web farm.

<
The issue with the calendar was just an example. The larger point is that a module's output might change due to
circumstances other than user action. Assuming you want to allow add-on modules to support caching, I think we need
a more general approach.
<

I understand, what other circumstances are you thinking of? How long do you think we should cache things?

>
At the moment, I'm thinking that SiteModuleControl should have an additional method
called GetVaryByCustomString().

Global.GetVaryByCustomString() would instantiate and configure the SiteModuleControls associated with the requested page
and would then call each SiteModuleControl's GetVaryByCustomString() method, appending the results.
<
SiteModuleControl is a web UserControl.
Unless you really mean Module.cs business class which will be managed inside SiteSettings, this seems just as expensive as not caching, if we have to do that I think we should skip the page output cache and just stick with the module cache where we can currently specify different cache times per module.

<
To handle the blog calendar case, BlogModule.GetVaryByCustomString() would return a string that contained, among other things, the current date. That would force a new page to be rendered each day.
<

Definitely the date can be part of the variance. We also must vary by culture.

8/31/2006 5:36:33 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I think we can add the last update to the Module class and let SiteSettings orchestrate everything. SiteSettings will cache the last update time of each module for at least 1 minute (thats a lot of requests if under heavy load)

There is some confusion in our terms and I know I'm guilty of using them interchangably. SiteModuleControl inherits from UserControl and is the base class for the .ascx file associated with each feature. Module.cs is the buisiness class it knows which .ascx control is needed, the roles, etc, but there is also a business class associated with each feature Blog.cs, HtmlContent.cs and these are loaded inside the SiteModuleControl and get the actual content from the db.

Currently we have caching at the SiteModule rendered content level with configurable cache time.

8/31/2006 6:52:18 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

what other circumstances [that might change a module's output] are you thinking of?

I can think of several other types of modules whose output would change even without a DB update. Consider an RSS aggregator, a stock ticker, a quote of the day, weather forecast, or more generally a front-end to some sort of web service.

How long do you think we should cache things?

Assuming we can limit total cache memory usage via the web.config, I think we should cache things until we know they will be invalid. But force the browser to revalidate so that the user always sees the latest version.

SiteModuleControl is a web UserControl.
Unless you really mean Module.cs business class which will be managed inside SiteSettings,

I meant SiteModuleControl not Module.cs. I originally thought that you could put all the output caching logic in the business objects, but then I realized that if you want developers to be able to add their own modules, you can't assume that such modules will generate the same HTML for a given set of business objects. A very common case would be a module that generates different HTML depending on the User-Agent header. You could also have a module that displays the user's IP address, or more likely, uses it to determine the user's physical location so that content can be better targeted at them.

The bottom line is that if you are caching the output of a SiteModuleControl, the SiteModuleControl needs to have some control over that caching. Output caching is intrinsically a presentation layer activity, so the logic controlling it belongs in that layer.

this seems just as expensive as not caching,

I suspect not because I think the bulk of the CPU is spent rendering and sending the page to the browser, not instantiating and configuring the SiteModuleControl objects. One easy way to test this hypothesis would be to measure the performance impact of duplicating the "foreach (Module module in siteSettings.ActivePage.Modules)" loop in Default.aspx.cs (removing the added controls after the first loop completes).

if we have to do that I think we should skip the page output cache and just stick with the module cache where we can currently specify different cache times per module.

For the record, I have no problem skipping output caching. In fact, I think you should rerun your tests of the benefits of module caching with a more simultaneous browser connections. You might find that even module caching doesn't buy as much as we thought. I suspect that the 2 performance bottlenecks are DB I/O and the CPU cycles required to send a rendered page to the browser. DB I/O might actually not be that big of an issue since the DB isn't that large and doesn't see many writes. That would mean that the primary way to improve performance would be to reduce the number of bytes sent to the browser. There are only 2 ways to do that. The first is to trim the fat in the HTML. The second is to cache on the browser (or someplace other than the server).

9/1/2006 12:53:37 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Yeah I'm kind of back to punting on the page output caching myself. I may take a stab at it later but I'm going to focus on trying to reduce page size as much as possible as you suggest.

How many browser sessions do you recommend using? My last tests above were with 4 browsers. How many should a I ramp it up to? When testing with Act on my laptop against the remote web server, the laptop wasn't working very hard. Is that what I should go by? Ramp it up until the Act machine is working at full capacity?

What I'd like to do is setup a dotnetnuke site, maybe community server site too and try creating comparable tests for each so that I can get an idea of how well we are performing compared to these more popular/well known products.

9/1/2006 2:34:21 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Repeatedly double num browser sessions up until server cpu pegs at 100% or "average time until first byte" roughly doubles. That should indicate that the server has hit a bottleneck. Make sense?

9/1/2006 2:39:59 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

The remote web server was pegged at 100% cpu in the last tests. Do you mean the ACT machine? It wasn't working very hard.

9/1/2006 3:53:24 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

No, I mean the remote web server. I don't think you need to redo the last tests. I think you should redo the earlier 2 tests where you weren't doing page output caching. One with and one without module caching. That will give you a better indication of what you are gaining with module caching and page output caching.

9/1/2006 4:08:20 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

The last 2 tests were with module caching and no page output caching. One was on running all locally on my laptop with both ACT and Web/SQL and one was with ACT running on my laptop and the Web/SQL running on a remote P3 800 machine.

Tests previous to those showed significant benefit to module caching vs not module caching and dumb pageoutput caching also showed significant improvement over module caching though that was run all on my laptop. I can redo the comparisons on the remote machine though thats no problem.
I'll run them in the morning and post the results.

9/2/2006 1:06:07 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

OK, here are some more tests run against a remote machine. I have not yet done anything to reduce page size in mojoPortal, so this is still just trying to establish a baseline before working on that and to get a comparisin vs DotNetNuke.

The ACT machine is my laptop Pentium M 2.13Ghz with 2Gigs of RAM
The Test web server is Pentium3 with 1Gig of RAM running Win2003 Server with MS SQL 2005 on the same machine.

In all tests the ACT machine was barely working and the web server was 100%cpu. 4 browsers seemed to be the right amount to push the processor to 100% on the web server.

In between each test I rebooted the web server and waited a few minutes to ensure all services were up and running.

I tested both mojoPortal, and the latest download of DotNetNuke, v4.3.4

The test sites consists of 5 pages, Home, Blog, Events, Forum, Image Gallery, each page has the corresponding module, the home page has an Html module and a links module

In both the mojoportal site and the DNN site I entered the exact same content in the modules.

mojoPortal using no Module Cache
maximum wpg memory use during test was 51,336K

mojoPortal using Module Cache of 360 Seconds
maximum wpg memory use during test was 49,024K

mojoPortal using dumb page output cache
maximum wpg memory use during the test was 47,952K
"dumb" means it renders the same page to all users regardless fo roles or anything else, it just caches the entire page and serves that copy up to all requests until the cache expires. We can't actually use this but it gives an idea of the upper bounds of possible improvement through caching.

DotNetNuke 4.3.4
The html module in DNN has a default cache time of 1200 seconds and the links module has a default of 60 seconds, the forums, blog, and image gallery all have defaults of 0 seconds which I assume is no cache.
For the first test I left things at the defaults.
Maximum wpg memory use during the first test was 100,644K

DotNetNuke test with additional modules cached
I then tried bumping up the cache time on the Blog, Events, Forum, and Image Gallery of DNN to 360 seconds.
Apparently there are some major problems in caching the Events or the Blog as the site became un-responsive to anonymous requests of the Blog page or Events page.
wpg memory consumption went up over 632,248K and seemed as if it would keep going up the longer the test was run.

Tests with the browser showed it was the blog and event pages that were un-responsive with anonymous sessions, though it seemed ok when logged in as admin, not sure whether caching is disabled or handled differently for admins or authenticated users but obviously setting a cache time greater than 0 on blgo module or events module in DNN will hose up a site when visited by anonymous browsers.

I had heard from a number of people that DNN has performance problems but now I see it is more than anecdotal, there are some performance issues and apparently memory leaks in the lates version of DNN.

9/2/2006 2:12:49 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Here's my quick and dirty analysis:

Processing of requests can be broken into three parts:

A. Rendering modules. This includes DB calls, business object instantiation, and rendering to HTML
B. Rendering a page from rendered modules.
C. Sending the rendered page to the browser.

A+B+C = 130 req/sec = 7.7 ms/req
B+C = 190 req/sec = 5.3 ms/req
C = 300 req/sec = 3.3 ms/req

Taking differences:

A = 2.4 ms/req = 31%
B = 2.0 ms/req = 26%
C = 3.3 ms/req = 43%

Reducing the number of bytes sent to the browser definitely seems to be the way to go. Not only will it reduce C but it will probably also reduce A and B because they both involve rendering those bytes.

--Dean

9/2/2006 3:10:52 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I think your analysis is right on. Here are the file sizes from the test when saved as html from IE

Obviously file size was a factor in the DNN tests since the files are bigger.
I can understand the dnnForum page being larger as it has some additional things compared to the mojoPortal Forum, like a scrolling marquee of the most recent posts. But in general I think there is heavier table layout markup in the DNN pages, maybe more viewstate too I haven't measured it.

Interesting since I entered the same content in both, yet there is still room for improvement in mojoPortal. I'll be working on this tomorrow and will post some results. I'll be testing after little changes to see if the page is smaller and if the performance of the tests goes up.

9/4/2006 6:30:04 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

After doing some work to reduce the size of viewstate and to reduce the amount of inline styles the file sizes were a good bit smaller and performance improved.

File Sizes from first test (same as previous post)

File sizes after reducing viewstate and inline styles

This is with the exact same content in the db as before.

mojoPortal Using no module cache

mojoPortal with All modules Cached 360 seconds

However, I discovered in my testing that not all modules in their current implementation can work correctly with caching enabled. Modules that use postback don't seem to work as expected. I tried working around this by checking whether its a postback in the CachedSiteModule control by always loading a fresh copy instead of the cached one on postback but then it did not seem to fire the events. Maybe I can figure out a way at some point but for now I disabled cache on the 2 modules affected, Blog and Events. Blog works fine except for the calendar navigation so it could be cached if not showing the calendar. Events was affected by the same problem because it is essentially a calender inheriting from the asp.net Calendar. Possibly these could be re-implemented at some point to use javascript calendars and querystring based rather than postback based navigation. So for current real world comparison I tested again after setting cache to 0 (no cache) on the blog and events modules. This configuration is comparable to the defaults in DotNetNuke, they cache Html module and links module by default but not Events and not Blog (but don't forget if you enable caching Blog and Events in DNN it causes serious memory leak and performance degradation).

mojoPortal with All Modules except Blog and Events cached 360 seconds

So, in summary, reducing page size by reducing viewstate and inline markup increased performance as follows:
Using no Module Cache                    RPS went from 123.90 to 135.65
Using Module Cache on all Modules     RPS went from 191.52 to 200.57

and caching all modules except Blog and Events came in at 171.76 RPS

The main trick to reducing inline styles was not using style properties from the theme.skin file and only setting css class names there. Using the properties for style seems to result in very verbose inline styles. For example the asp.net Calendar control the original theme file was like this:

<asp:Calendar runat="server"
     BackColor=""
     BorderColor=""
     BorderStyle="solid"
     BorderWidth="0"
     CaptionAlign="Top"
     CellPadding="3"
     CellSpacing="0"
     CssClass=""
     DayHeaderStyle-BackColor=""
     DayHeaderStyle-BorderColor=""
     DayHeaderStyle-BorderStyle="Solid"
     DayHeaderStyle-BorderWidth="0"
     DayHeaderStyle-CssClass=""
     DayHeaderStyle-Font-Bold="true"
     DayHeaderStyle-Font-Italic="false"
     DayHeaderStyle-Font-Names=""
     DayHeaderStyle-Font-Overline="false"
     DayHeaderStyle-Font-Size="8pt"
     DayHeaderStyle-Font-Strikeout="false"
     DayHeaderStyle-Font-Underline="false"
     DayHeaderStyle-ForeColor="black"
     DayHeaderStyle-Height=""
     DayHeaderStyle-HorizontalAlign="center"
     DayHeaderStyle-VerticalAlign="NotSet"
     DayHeaderStyle-Width=""
     DayHeaderStyle-Wrap="false"
     DayNameFormat="FirstLetter"
     DayStyle-BackColor=""
     DayStyle-BorderColor=""
     DayStyle-BorderStyle="solid"
     DayStyle-BorderWidth="0"
     DayStyle-CssClass=""
     DayStyle-Font-Bold="false"
     DayStyle-Font-Italic="false"
     DayStyle-Font-Names=""
     DayStyle-Font-Overline="false"
     DayStyle-Font-Size="8pt"
     DayStyle-Font-Strikeout="false"
     DayStyle-Font-Underline="false"
     DayStyle-ForeColor="black"
     DayStyle-Height=""
     DayStyle-HorizontalAlign="center"
     DayStyle-VerticalAlign="NotSet"
     DayStyle-Width=""
     DayStyle-Wrap="false"
     FirstDayOfWeek="sunday"
     Font-Bold="false"
     Font-Italic="false"
     Font-Names=""
     Font-Overline="false"
     Font-Size="small"
     Font-Strikeout="false"
     Font-Underline="false"
     ForeColor="Black"
     Height=""
     NextMonthText="+"
     NextPrevFormat="CustomText"
     NextPrevStyle-BackColor=""
     NextPrevStyle-BorderColor="black"
     NextPrevStyle-BorderStyle="solid"
     NextPrevStyle-BorderWidth="0"
     NextPrevStyle-CssClass=""
     NextPrevStyle-Font-Bold="true"
     NextPrevStyle-Font-Italic="false"
     NextPrevStyle-Font-Names=""
     NextPrevStyle-Font-Overline="false"
     NextPrevStyle-Font-Size="small"
     NextPrevStyle-Font-Strikeout="false"
     NextPrevStyle-Font-Underline="false"
     NextPrevStyle-ForeColor="black"
     NextPrevStyle-Height=""
     NextPrevStyle-HorizontalAlign="center"
     NextPrevStyle-VerticalAlign="NotSet"
     NextPrevStyle-Width=""
     NextPrevStyle-Wrap="false"
     OtherMonthDayStyle-BackColor="#f3f3f3"
     OtherMonthDayStyle-BorderColor="black"
     OtherMonthDayStyle-BorderStyle="solid"
     OtherMonthDayStyle-BorderWidth="0"
     OtherMonthDayStyle-CssClass=""
     OtherMonthDayStyle-Font-Bold="false"
     OtherMonthDayStyle-Font-Italic="false"
     OtherMonthDayStyle-Font-Names=""
     OtherMonthDayStyle-Font-Overline="false"
     OtherMonthDayStyle-Font-Size="8pt"
     OtherMonthDayStyle-Font-Strikeout="false"
     OtherMonthDayStyle-Font-Underline="false"
     OtherMonthDayStyle-ForeColor="black"
     OtherMonthDayStyle-Height=""
     OtherMonthDayStyle-HorizontalAlign="center"
     OtherMonthDayStyle-VerticalAlign="NotSet"
     OtherMonthDayStyle-Width=""
     OtherMonthDayStyle-Wrap="false"
     PrevMonthText="-"
     SelectedDayStyle-BackColor="Beige"
     SelectedDayStyle-BorderColor="black"
     SelectedDayStyle-BorderStyle="solid"
     SelectedDayStyle-BorderWidth="0"
     SelectedDayStyle-CssClass=""
     SelectedDayStyle-Font-Bold="false"
     SelectedDayStyle-Font-Italic="false"
     SelectedDayStyle-Font-Names=""
     SelectedDayStyle-Font-Overline="false"
     SelectedDayStyle-Font-Size="8pt"
     SelectedDayStyle-Font-Strikeout="false"
     SelectedDayStyle-Font-Underline="false"
     SelectedDayStyle-ForeColor="black"
     SelectedDayStyle-Height=""
     SelectedDayStyle-HorizontalAlign="center"
     SelectedDayStyle-VerticalAlign="NotSet"
     SelectedDayStyle-Width=""
     SelectedDayStyle-Wrap="true"
     SelectorStyle-BackColor="#eee"
     SelectorStyle-BorderColor="red"
     SelectorStyle-BorderStyle="solid"
     SelectorStyle-BorderWidth="0"
     SelectorStyle-CssClass=""
     SelectorStyle-Font-Bold="true"
     SelectorStyle-Font-Italic="false"
     SelectorStyle-Font-Names=""
     SelectorStyle-Font-Overline="false"
     SelectorStyle-Font-Size="8pt"
     SelectorStyle-Font-Strikeout="false"
     SelectorStyle-Font-Underline="false"
     SelectorStyle-ForeColor="Green"
     SelectorStyle-Height=""
     SelectorStyle-HorizontalAlign="center"
     SelectorStyle-VerticalAlign="NotSet"
     SelectorStyle-Width=""
     SelectorStyle-Wrap="false"
     ShowDayHeader="true"
     ShowGridLines="true"
     ShowNextPrevMonth="true"
     ShowTitle="true"
     TitleFormat="MonthYear"
     TitleStyle-BackColor=""
     TitleStyle-BorderColor="black"
     TitleStyle-BorderStyle="solid"
     TitleStyle-BorderWidth="0"
     TitleStyle-CssClass=""
     TitleStyle-Font-Bold="true"
     TitleStyle-Font-Italic="false"
     TitleStyle-Font-Names=""
     TitleStyle-Font-Overline="false"
     TitleStyle-Font-Size="8pt"
     TitleStyle-Font-Strikeout="false"
     TitleStyle-Font-Underline="false"
     TitleStyle-ForeColor="Black"
     TitleStyle-Height=""
     TitleStyle-HorizontalAlign="center"
     TitleStyle-VerticalAlign="NotSet"
     TitleStyle-Width=""
     TitleStyle-Wrap="false"
     TodayDayStyle-BackColor="lightyellow"
     TodayDayStyle-BorderColor="black"
     TodayDayStyle-BorderStyle="solid"
     TodayDayStyle-BorderWidth="0"
     TodayDayStyle-CssClass=""
     TodayDayStyle-Font-Bold="false"
     TodayDayStyle-Font-Italic="false"
     TodayDayStyle-Font-Names=""
     TodayDayStyle-Font-Overline="false"
     TodayDayStyle-Font-Size="8pt"
     TodayDayStyle-Font-Strikeout="false"
     TodayDayStyle-Font-Underline="false"
     TodayDayStyle-ForeColor="Black"
     TodayDayStyle-Height=""
     TodayDayStyle-HorizontalAlign="center"
     TodayDayStyle-VerticalAlign="NotSet"
     TodayDayStyle-Width=""
     TodayDayStyle-Wrap="true"
     WeekendDayStyle-BackColor="lightgray"
     WeekendDayStyle-BorderColor="black"
     WeekendDayStyle-BorderStyle="solid"
     WeekendDayStyle-BorderWidth="0"
     WeekendDayStyle-CssClass=""
     WeekendDayStyle-Font-Bold="false"
     WeekendDayStyle-Font-Italic="false"
     WeekendDayStyle-Font-Names=""
     WeekendDayStyle-Font-Overline="false"
     WeekendDayStyle-Font-Size="8pt"
     WeekendDayStyle-Font-Strikeout="false"
     WeekendDayStyle-Font-Underline="false"
     WeekendDayStyle-ForeColor="black"
     WeekendDayStyle-Height=""
     WeekendDayStyle-HorizontalAlign="center"
    WeekendDayStyle-VerticalAlign="NotSet"
     WeekendDayStyle-Width=""
     WeekendDayStyle-Wrap="true"
     Width=""
    />

and after moving the style settings into separate .css file the theme.skin was like this:

<asp:Calendar runat="server"
     CaptionAlign="Top"
     CssClass="aspcalendarmain"
     DayHeaderStyle-CssClass="aspcalendardayheader"
     DayNameFormat="FirstLetter"
     DayStyle-CssClass="aspcalendarday"
     FirstDayOfWeek="sunday"
     NextMonthText="+"
     NextPrevFormat="CustomText"
     NextPrevStyle-CssClass="aspcalendarnextprevious"
     OtherMonthDayStyle-CssClass="aspcalendarothermonth"
     PrevMonthText="-"
     SelectedDayStyle-CssClass="aspcalendarselectedday"
     SelectorStyle-CssClass="aspcalendarselector"
     ShowDayHeader="true"
     ShowGridLines="false"
     ShowNextPrevMonth="true"
     ShowTitle="true"
     TitleFormat="MonthYear"
     TitleStyle-CssClass="aspcalendartitle"
     TodayDayStyle-CssClass="aspcalendartoday"
     WeekendDayStyle-CssClass="aspcalendarweekendday"

    />

9/4/2006 6:56:48 AM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Performance improvement is substantially less than I would have expected. Not sure why. You said the test involve visiting each of the 5 pages, but the ACT results show 1194 iterations, 40695 total requests, and 28 unique requests. Any idea why? Is ACT requesting the images on each page in addition to the HTML? Somethinge else?

--Dean

9/4/2006 7:00:17 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Yes, the ACT script seems to request each file associated with each page, images, css, js etc.
Here is an example fragment from the test script that ACT generated:

Sub SendRequest7()
    Dim oConnection, oRequest, oResponse, oHeaders, strStatusCode
    If fEnableDelays = True then Test.Sleep (31)
    Set oConnection = Test.CreateConnection("loadtest", 80, false)
    If (oConnection is Nothing) Then
        Test.Trace "Error: Unable to create connection to loadtest"
    Else
        Set oRequest = Test.CreateRequest
        oRequest.Path = "/mojo/Data/Sites/1/logos/mojotonguesmall.gif"
        oRequest.Verb = "GET"
        oRequest.HTTPVersion = "HTTP/1.0"
        set oHeaders = oRequest.Headers
        oHeaders.RemoveAll
        oHeaders.Add "Accept", "*/*"
        oHeaders.Add "Referer", "http://loadtest/mojo/"
        oHeaders.Add "Accept-Language", "en-us,it;q=0.91,de;q=0.82,nl;q=0.73,pt-br;q=0.64,ru;q=0.55,cs;q=0.45,ar-iq;q=0.36,es-mx;q=0.27,tr;q=0.18,es;q=0.09"
        oHeaders.Add "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; InfoPath.1)"
        'oHeaders.Add "Host", "loadtest"
        oHeaders.Add "Host", "(automatic)"
        'oHeaders.Add "Cookie", ".ASPXANONYMOUS=2lksUHn1XnRaup90RPMBynbWw2f7745q0xZRq2QnY85H4eCWQ6NudN20yphIUdM2Y27QTlSSxVc-2uShJkiKsj6mNGoVDbJ80; DisplayName=Admin; ASP.NET_SessionId=ouu1rz55zo0ivwexohawa4bm"
        oHeaders.Add "Cookie", "(automatic)"
        Set oResponse = oConnection.Send(oRequest)
        If (oResponse is Nothing) Then
            Test.Trace "Error: Failed to receive response for URL to " + "/mojo/Data/Sites/1/logos/mojotonguesmall.gif"
        Else
            strStatusCode = oResponse.ResultCode
        End If
        oConnection.Close
    End If
End Sub

Joe

9/4/2006 8:13:08 AM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

OK. So we need to add:

D. Sending static content to the browser.

and the test you've been running so far assumes that all users are new users. To measure performance for existing users, you should probably add:

oHeaders.Add "If-Modified-Since", "Mon, 4 Sep 2006 00:00:00 GMT"

to each of the requests for static content. I'm assuming that ACT doesn't do that automatically somehow. Note that if you do that, you'll either need to update that data manually if you modify the static content, or generate it automatically in the correct format. A simpler method would be to just not make the requests for static content at all. For that to be truly realistic though mojoPortal would need to provide a way to offload the static requests to a different server, a la akamai. I don't think that would be that hard to implement. Basically wherever you generate URLs that contain "Data/Sites", you'd allow for a different hostname to be used.

Also, it's not clear whether ACT is actually creating a new connection for each request or whether it is pooling connections somehow. If it's creating a new connection that could create a slightly higher load on the server and increase the time to first byte. I'd ignore that for now though.

--Dean

9/4/2006 8:55:16 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Yeah, thats what I meant before about not being sure whether ACT would measure the benefit of browser caching. I think I need to spend some time just reading up on ACT to see how it can be configured for various testing scenarios but for now I've got at least a day of work continuing to prune the theme.skin files and add css classes for each of the built in skins.

I think the main thing that is helpful is being able to measure whether particular changes degrade or improve performance. I do want to see if we can leverage browser caching though so I will read up on this after I finish the theme/css work.

9/10/2006 1:01:25 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I learned from my friend David Neal an interesting fact about load testing with ACT. He was using it at work this week in conjunction with Ants Profiler and he made an interesting discovery.

When you record a browser session to create the test, it creates separate requests for each image, .js, .css, etc as we already know from my previous tests. The interesting thing is that these requests actually improve measured performance dramatically because they are served up directly by IIS and not handled by .NET, they are served up very quickly and their inclusion boosts the number of requests per second quite a bit. I haven't tried this yet myself but David said when he commented those out the RPS went down to single digits in the app he was testing. So if you really want to see how fast the server code is running and be able to tell if changes are helping or hurting its probably better to comment out the static file requests.

I had a busy weekend developing and didn't get to trying it myself but hopefully will in the next night or so.

9/12/2006 5:28:55 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I finally got back to trying the tests with just the .aspx pages as my friend David suggested. Eliminating the requests for .css, .js, image files etc reduced both tests (mojoPortal and DotNetNuke) to 7 requests each test iteration. For the mojoPortal results it seems that ACT doesn't consider query string params to count toward unique requests so it counted / and /Default.aspx whereas with DNN the pattern is /tabid/n/Default.aspx where n is the tab id so it counts each one as unique. However both tests consisted of 7 requests each.

The results confirm that the requests for the static files improve the average RPS as both mojoPortal and DotNetNuke dropped RPS substantially. However mojoPortal still performed quite a bit better than DotNetNuke.

in summary:
mojoPortal 41.94 RPS
vs
DotNetNuke 10.46 RPS

mojoPortal .aspx requests only

DotNetNuke .aspx requests only

9/12/2006 9:00:56 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Updated analysis (assuming your latest test was with no caching and the server was using 100% CPU):

A. Rendering modules. This includes DB calls, business object instantiation, and rendering to HTML
B. Rendering a page from rendered modules.
C. Sending the rendered page to the browser.
D. Sending static content to the browser.

A+B+C+D = 251 ms/iter
B+C+D = 170 ms/iter
A+B+C = 167 ms/iter

Taking differences:

A = 81 ms/iter = 29%
B+C = 86 ms/iter = 34%
D = 84 ms/iter = 33%

To split out B and C, you'd need to do another test with "dumb output caching", though I suspect that C is pretty neglible. If so, it looks like module caching will roughly double performance (neglecting static content). To improve beyond that it looks like output caching would be needed.

--Dean

9/13/2006 3:28:10 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

Actually, this last test was using module caching in all modules except blog and events for mojoportal and using default module caching in DNN. Sorry, I should have indicated that. I'll do another round soon to measure the benefit of caching vs non-caching. This was just to make sure my comparison to DNN was fair.

Joe

9/13/2006 4:06:20 AM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

OK. So, the correct analysis would be (note redefinition of A and B below):

A. Rendering modules other than blog and events. This includes DB calls, business object instantiation, and rendering to HTML
B. Rendering blog and events and rendering a page from rendered modules.
C. Sending the rendered page to the browser.
D. Sending static content to the browser.

A+B+C+D = 251 ms/iter
B+C+D = 170 ms/iter
B+C = 167 ms/iter

Taking differences:

A = 81 ms/iter = 32%
B+C = 167 ms/iter = 67%
D = 3 ms/iter = 1%

I suspect most of the time spent in B+C is spent rendering the blog and events modules. It might be worthwhile to do a test where you cache those even if the cached HTML is somewhat broken. That would give you a feel for the max you can gain from module caching.

--Dean

9/13/2006 4:27:33 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

OK, so the last test was just .aspx pages (no static file requests) using reasonable module caching, that is, caching modules that can be cached without impacting their functionality.

Here are the mojoPortal results for

No caching
Caching all modules, even blog and events calendar which don't function correctly with caching. This is to establish the upper bound of improvement that can be achieved with module caching.
Dumb page output caching. This also isn't appropriate for real world but establishes an upper bound of improvement that could be achieved with page output caching.

Again this is testing using ACT on my laptop against a remote machine P3 800Mhz with 1GB RAM running IIS and MS SQL

mojoPortal .aspx pages with no cache

mojoPortal .aspx pages with all modules cached

mojoPortal .aspx pages with dumb page output caching

9/13/2006 4:49:01 PM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I thought I might try some testing with this http compression module:
http://www.blowery.org/code/HttpCompressionModule.html

I've downloaded it and added it to svn the other night but haven't tried it yet.

I'll post some more results after I get a chance to play with it.

Joe

9/13/2006 7:21:14 PM

Dean Brettle

Total Posts 148

View Posts

Re: SiteSetting class need redesign

Updated analysis (switching back to original definitions for A and B):

A. Rendering modules. This includes DB calls, business object instantiation, and rendering to HTML
B. Rendering a page from rendered modules.
C. Sending the rendered page to the browser.
D. Sending static content to the browser.

A+B+C+D = 251 ms/iter
A+B+C = 195 ms/iter
B+C = 139 ms/iter
C = 80 ms/iter

Taking differences:

A = 56 ms/iter = 22%
B = 59 ms/iter = 24%
C = 80 ms/iter = 32%
D = 56 ms/iter = 22%

Also:

dynamic + static content = 195 KB/iter
static content = 87 KB/iter

So:

dynamic content = 108 KB/iter = 1.35 MB/sec
static content = 87 KB/iter = 1.55 MB/sec

Since those numbers are well below typical processor<->memory bandwidths, the bottleneck is probably the network stack. That means using the HttpCompressionModule might very well help reduce time spent in C (and maybe D if you don't try to compress static images). Of course it all depends on the speed at which the compression can be performed and the reduction in size of the content.

--Dean

10/31/2006 4:21:11 AM

Joe Audette

Total Posts 18439

View Posts

Re: SiteSetting class need redesign

I just came accross an interesting article that shows another technique for injecting some dynamic content into a page or control that uses outputcache
http://www.dotnetbips.com/articles/displayarticle.aspx?id=516

The Substitution control, I wasn't aware of this control at all, though now I see it is even listed in the book I have, just didn't notice it.

I may have to do some more experimenting with this and see if I can overcome the issues of output caching using this.

Joe

This forum is for discussing mojoPortal development

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign

Re: SiteSetting class need redesign