Web based programming

[keith.lamothe]

If you have a different thread in mind please point me there.

I wrote everything using IHttpHandler and not much else

I remember seeing a framework like that somewhere and wanted to give it a try, anything you can point me at for bypassing the Page class?

I don't use their session state

I currently have my own as well but I was thinking of falling back on their Session object because it works with their Out-of-proc session management and the In-proc (where all static scope stuff is) dies every time we change any of the dlls.  In my environment we often make dozens of code changes in the production environment every day, and everyone on the server losing their sessions dozens of times a day...  but I could just do a db session and key it with the client mem-cookie that doesn't care about my dlls... hmm...

I also don't use their viewstate

That was the first thing to go for me.  I don't know if they've changed it yet but some of those data-bound controls actually *trust* the data coming back through viewstate when updating a record.  I'm hoping I misunderstood that part, but even with security aside I didn't want that stuff in there.

I don't have WYSIWYG support in my classes

We never use it either, even though we've used Dreamweaver as the ColdFusion editor due to syntax highlighting and partial "intellisense" like functionality.  I honestly wish I could disable Dreamweaver's design tab because it pops up sometimes unbidden.

This is a half a million line of code system

Not sure what ours is up to; the core isn't that big, about 550KB of source code.  A huge part of it is generating a ton of classes for each db table/column for each schema we develop, and the gen'd source for just two of the core DBs we use comes to ~4.3MB.

so it's more important to be able to (...) enable reflection, etc.

I've gone out of my way to avoid the need for normal app code to use reflection, so barely any dynamic casts and tons of fairly complex constrained generic types (though most of the concrete classes don't need type parameters).  My main concern was maintaining static analysis of the code so we don't get runtime type incompatibilities and whatnot (not that I have any right to complain about late binding when coming from ColdFusion) and secondarily I was concerned about performance. 

That's also one of the reasons I'm not including LINQ: how do I control the number of trips it takes to the DB server, because those network connections take way more time than anything else?  Functionally I love LINQ, but sometimes I really want to control _how_ it does stuff.

Anyway, I should sign off for the night, thanks for the discussion.
Keith

[x4000]

Okay, that splits this out and solves that issue.  I'll respond to this in the morning, I'm signing off for the night.  Cheers!

[x4000]

I remember seeing a framework like that somewhere and wanted to give it a try, anything you can point me at for bypassing the Page class?

It's been about four years since I wrote that, so I don't remember what references I had at the time, but here's a reference that shows how to do essentially what I've done:  http://www.brainbell.com/tutorials/ASP/Implementing_IHttpHandler.html.  Overall, it's really a very easy process, the hard part is then just designing your internal control structure that lets you emit HTML.  I've basically got mine organized into an overall Page class, and then things WebLabel, WebTextbox, WebDateBox, WebEmailBox, WebListing, WebListBox, WebCheckbox, etc -- I think I have around 60-70 classes in all by now with that.  That's the part that takes time, but you can make them super efficient and just do exactly what you want, which is cool if you're designing and coding a very long term system (or a framework for multiple systems).

I don't use their session state

I currently have my own as well but I was thinking of falling back on their Session object because it works with their Out-of-proc session management and the In-proc (where all static scope stuff is) dies every time we change any of the dlls.  In my environment we often make dozens of code changes in the production environment every day, and everyone on the server losing their sessions dozens of times a day...  but I could just do a db session and key it with the client mem-cookie that doesn't care about my dlls... hmm...

Yeah, using cookies or querystring you can have a session key (we have a set of three), and then it is always something you can look up in the database.  This is the only way you can really do load balancing, as well.  We have a cluster of front-end web servers, so that means that using anything in-memory that is not for building the current page is not possible, anyway.  So that let's us swap in updated dlls whenever we want without interrupting users (and we frequently do).

I also don't use their viewstate

That was the first thing to go for me.  I don't know if they've changed it yet but some of those data-bound controls actually *trust* the data coming back through viewstate when updating a record.  I'm hoping I misunderstood that part, but even with security aside I didn't want that stuff in there.

Yeah, it's pretty crazy, and also so bloated.

I don't have WYSIWYG support in my classes

We never use it either, even though we've used Dreamweaver as the ColdFusion editor due to syntax highlighting and partial "intellisense" like functionality.  I honestly wish I could disable Dreamweaver's design tab because it pops up sometimes unbidden.

Ah, yeah -- I use much the same thing in Visual Studio.  In our case we have no raw html at all, just .NET classes that emit html based on our internal control structure.  So it's all with strings, etc, in C#, and thus we get full intellisense and no design tab.

This is a half a million line of code system

Not sure what ours is up to; the core isn't that big, about 550KB of source code.  A huge part of it is generating a ton of classes for each db table/column for each schema we develop, and the gen'd source for just two of the core DBs we use comes to ~4.3MB.

Ah, that makes sense, I see you are going with an OOP style database interface.  We are all about the stored procedures, and handle our TSQL in a very different way from the upper-level C#.  With all the crazy reports and aggregation and stuff that we have to do, that's the only feasible thing for us -- have to utilize the strengths of each platform, etc, etc.  But I've seen the method you're using in the past, and have been intrigued by it.  I'm too much of a TSQL junkie to ever really go that route, though, most likely.

so it's more important to be able to (...) enable reflection, etc.

I've gone out of my way to avoid the need for normal app code to use reflection, so barely any dynamic casts and tons of fairly complex constrained generic types (though most of the concrete classes don't need type parameters).  My main concern was maintaining static analysis of the code so we don't get runtime type incompatibilities and whatnot (not that I have any right to complain about late binding when coming from ColdFusion) and secondarily I was concerned about performance. 

We use reflection to enable dynamic class loading.  That's a lot more efficient for us (with 2700+ pages in the system) than having a big factory method or something.  We basically have one "page" in the system (rh.sh), and then a querystring key of "content" tells it what class to load via reflection.  There are a few other uses for it that also really ease development for us.  We don't do much in the way of late binding aside from this, so we don't get anything else beyond the expected "class not found" errors if someone makes an erroneous request.

That's also one of the reasons I'm not including LINQ: how do I control the number of trips it takes to the DB server, because those network connections take way more time than anything else?  Functionally I love LINQ, but sometimes I really want to control _how_ it does stuff.

Oh, lord I would NEVER use LINQ with a database.  I'm with you there.  That's why I'm all about the TSQL, I totally love that control also.  I really am just a fan of LINQ when it comes to interrogating in-memory classes and lists.

Anyway, I should sign off for the night, thanks for the discussion.
Keith

You bet!  Thank you, as well.
Chris

[keith.lamothe]

Chris,

Thanks for the link to the IHttpHandler tutorial.  So I'd have to change the IIS settings to redirect the desired file extension to that dll?  I might be able to get the server admins to agree to that, but I should probably stick with my almost-completely-subverted ASP.NET Page-class based approach for now.  Later I could do some profiling to see how much overhead we could save with a custom http handler.

And the HTML emission wouldn't be a big deal because I'm already doing my own thing on that point and it would work just as fine in a custom handler as in the ASP-Page wrapper.  Basically we're so used to complete string-level control in both HTML and SQL emission that we have to maintain that for now until we find a control structure we really like.  There are tons of functions and optional control-ish classes that allow the developer to avoid that string-level control if they don't want it (and it's more or less mandatory that they avoid wide-scope string-level SQL control), but it's still pretty direct.  Since the standard .NET string type is immutable we wrote our own tree-string sort of class to maintain constant time-and-memory complexity on insert.  That wouldn't be necessary in a completely control-class-structure but with our way we don't have to have any concept of a list or tree of controls.  Basically I'm sure our approach is very non-optimal but we're mostly recovering coldfusion programmers so we can't change our patterns too fast

Good point on the load balancing, that's a pretty conclusive case to go with db-sessions because my boss would really like to be able to scale up the hardware a lot in case a client had the need and the dollars.

I see you are going with an OOP style database interface.

Sort of; a select query is like this:

PERSONNEL_Record personnelRecords;
{
   PERSONNEL_CriteriaStructure criteria = new PERSONNEL_CriteriaStructure();
   criteria.ID_eq.Add(pk_id);
   PERSONNEL_Query query = new PERSONNEL_Query(criteria);
   personnelRecords = query.SingleRecord;
}

(the { and } are optional, they just scope the criteria and query variables so I can reuse the names in a later sibling scope with different types and they aren't used in the parent scope)

We don't do stored procedures because we want to keep the code all at one layer where at all possible.  Do you know of a decent way to keep visibility (and versioning) of all stored procedures on a db server?  Also, we tend to need really insane reporting flexibility that pretty much requires completely dynamic sql ability (and thus a ton of injection prevention, but we've done pretty good with that).  I suppose we could build the dynamic sql string and send it to a stored procedure that runs executeQuery or whatever it is but that's still injection vulnerable.

We use reflection to enable dynamic class loading.

Yea, that's the one area I've really been thinking of doing that.  As it is we have to have a singleton class for each separate application and a router function in each app that matches the url "PageName" string to an actual ApplicationPage subclass (just a bunch of if-else-elseif).  Part of me kind of likes that, because it's easy to statically determine which page classes could possibly be used, whereas with dynamic class loading we'd have to assume that any ApplicationPage subclass anywhere could be used.

I really am just a fan of LINQ when it comes to interrogating in-memory classes and lists.

Yea, it is good for that.  I also used a LINQ-to-SQL mapping for our internal code generator app to access the MSSQL metadata because it just works, and it doesn't have to be blazingly fast or scale to thousands of concurrent requests.

The hardest problem we've had with ASP.NET is figuring out how to deploy dll changes to multiple disjoint websites.  In coldfusion you can just include a code library from anywhere on the file system as long as you map that directory in the CFAdmin (so changes to those libraries instantly manifest in all apps that reference them regardless of app location), but in ASP it insists that the bin folder be inside the IIS website's directory structure.  We could probably do a hard link (symbolic link) at the file system from each website/bin/ folder to a central dll repository folder, but I'm wary of doing such systems-level things for such a problem.  Our temporary solution while we figure out what we want to do is to just have a quasi-service that copies any changed dlls from a central dll folder to each of the "target" bin folders.  Any ideas on a cleaner solution?  It'd be really nice if the web.config file would let you specify a bin directory path outside the website root...

Thanks,
Keith

[x4000]

So I'd have to change the IIS settings to redirect the desired file extension to that dll?  I might be able to get the server admins to agree to that, but I should probably stick with my almost-completely-subverted ASP.NET Page-class based approach for now.  Later I could do some profiling to see how much overhead we could save with a custom http handler.

Only if you use a custom extension (I am using .sh).  If you continue to use the aspx extension, which is possible (I've done it), then the IIS settings don't need to be updated.  Just your web.config, if I recall.

And the HTML emission wouldn't be a big deal because I'm already doing my own thing on that point and it would work just as fine in a custom handler as in the ASP-Page wrapper.  Basically we're so used to complete string-level control in both HTML and SQL emission that we have to maintain that for now until we find a control structure we really like.  There are tons of functions and optional control-ish classes that allow the developer to avoid that string-level control if they don't want it (and it's more or less mandatory that they avoid wide-scope string-level SQL control), but it's still pretty direct.  Since the standard .NET string type is immutable we wrote our own tree-string sort of class to maintain constant time-and-memory complexity on insert.  That wouldn't be necessary in a completely control-class-structure but with our way we don't have to have any concept of a list or tree of controls.  Basically I'm sure our approach is very non-optimal but we're mostly recovering coldfusion programmers so we can't change our patterns too fast

No worries, there is a lot of flexibility to that approach.  The main thing that a control tree gives you is consistency.  I can make an upgrade or change to something, and it changes that control on every page in the entire system, which is really a huge benefit.  But I don't expect everyone to use that sort of approach, because it took a ton of upfront development work for us.

Good point on the load balancing, that's a pretty conclusive case to go with db-sessions because my boss would really like to be able to scale up the hardware a lot in case a client had the need and the dollars.

Yep, that was my thought too.  My software is web-based SaaS that supports many clients, so for us the load balancing was non-negotiable and we had to decide what to do there.

I see you are going with an OOP style database interface.

Sort of; a select query is like this:

PERSONNEL_Record personnelRecords;
{
   PERSONNEL_CriteriaStructure criteria = new PERSONNEL_CriteriaStructure();
   criteria.ID_eq.Add(pk_id);
   PERSONNEL_Query query = new PERSONNEL_Query(criteria);
   personnelRecords = query.SingleRecord;
}

(the { and } are optional, they just scope the criteria and query variables so I can reuse the names in a later sibling scope with different types and they aren't used in the parent scope)

Very cool, I've seen stuff like that before but never in an actual live system. Nice to know that someone I know is really using that approach, since it seems like it does have many benefits.

We don't do stored procedures because we want to keep the code all at one layer where at all possible.  Do you know of a decent way to keep visibility (and versioning) of all stored procedures on a db server?  Also, we tend to need really insane reporting flexibility that pretty much requires completely dynamic sql ability (and thus a ton of injection prevention, but we've done pretty good with that).  I suppose we could build the dynamic sql string and send it to a stored procedure that runs executeQuery or whatever it is but that's still injection vulnerable.

We use RedGate SQLCompare for all of our versioning stuff, but it's not the same as having source control or something.  It's less ideal than, say, SVN, but that's the only way to do it that we really like. MS has their own way of adding SourceSafe support for SQL 2005/2008, but I didn't like it.  We use basically the three-tier code approach to our application, with the database being the bottom tier and containing all of that code.  It's mostly a matter of preference, and which languages you prefer for which tasks, IMO.

We also have a lot of dynamic SQL needs for things like reports, but we have a couple of centralized C# classes that basically build the SQL statements and send them out kind of like how we are emitting html for the clients.  But that's really a minority of our code, we've got only two major groups of that code, and almost 3,000 SPs (and these are not "atomic" SPs, but rather much more involved ones that are sometimes hundreds or thousands of lines of code).  All a matter of preference, and what the design parameters are for your specific system.  I'm pretty convinced there is no one best way to build software, although there are many ways with varying advantages.

We use reflection to enable dynamic class loading.

Yea, that's the one area I've really been thinking of doing that.  As it is we have to have a singleton class for each separate application and a router function in each app that matches the url "PageName" string to an actual ApplicationPage subclass (just a bunch of if-else-elseif).  Part of me kind of likes that, because it's easy to statically determine which page classes could possibly be used, whereas with dynamic class loading we'd have to assume that any ApplicationPage subclass anywhere could be used.

We automatically append most of the start of the class name ("Starta.Web.Content."), so that prevents people from tryng to load classes outside of that (and reflection would fail, anyway).  I just can't stomach the maintenance of a big singleton factory class like you describe.  I did that in the past with a Mario clone platformer I was coding in 2003 (unreleased, of course), and it was a real nightmare after a few months.  But again, it's all personal preference.

I really am just a fan of LINQ when it comes to interrogating in-memory classes and lists.

Yea, it is good for that.  I also used a LINQ-to-SQL mapping for our internal code generator app to access the MSSQL metadata because it just works, and it doesn't have to be blazingly fast or scale to thousands of concurrent requests.

Ah, gotcha.  I could see that working well, too.

The hardest problem we've had with ASP.NET is figuring out how to deploy dll changes to multiple disjoint websites.  In coldfusion you can just include a code library from anywhere on the file system as long as you map that directory in the CFAdmin (so changes to those libraries instantly manifest in all apps that reference them regardless of app location), but in ASP it insists that the bin folder be inside the IIS website's directory structure.  We could probably do a hard link (symbolic link) at the file system from each website/bin/ folder to a central dll repository folder, but I'm wary of doing such systems-level things for such a problem.  Our temporary solution while we figure out what we want to do is to just have a quasi-service that copies any changed dlls from a central dll folder to each of the "target" bin folders.  Any ideas on a cleaner solution?  It'd be really nice if the web.config file would let you specify a bin directory path outside the website root...

What about putting your dlls into the GAC?  That's just the sort of thing the GAC is supposed to be used for.

Chris

[keith.lamothe]

I can make an upgrade or change to something, and it changes that control on every page in the entire system, which is really a huge benefit.

That is huge; what we'll do for now is encourage the developers to use the HTML "standard" emission functions and classes, and ask them to make a tagged comment note where they have to deviate from the standard so we can review for extension/parameterization of the standard stuff to meet that need.

I just can't stomach the maintenance of a big singleton factory class like you describe.

It doesn't scale well past a certain point, but the singleton will only have one entry per app, and we only add 20 apps in a good *year* so I'm not too worried about it.  The individual page classes aren't singletons, that would be a nastiness.  Basically I would have used an enum for the apps if C#'s enum support were as nice as Java's, but such as it is...

What about putting your dlls into the GAC?  That's just the sort of thing the GAC is supposed to be used for.

I already looked long and hard at the GAC, but if I'm understanding it correctly any new dll would have to be registered on the web server itself through IIS or a similar management console.  Doesn't it also have to be re-registered on a change for that change to take effect?  Not sure about that one.  Basically I want to be able to make a change to the source code and have the rebuild done and testing started in less than 10 seconds (preferably <5), and messing with the GAC doesn't sound like it'd be in that ballpark.

Or can I just drop new/changed dlls in the GAC folder and it picks them up like dlls dropped in the bin folder?

Thanks,
Keith

[x4000]

I just can't stomach the maintenance of a big singleton factory class like you describe.

It doesn't scale well past a certain point, but the singleton will only have one entry per app, and we only add 20 apps in a good *year* so I'm not too worried about it.  The individual page classes aren't singletons, that would be a nastiness.  Basically I would have used an enum for the apps if C#'s enum support were as nice as Java's, but such as it is...

Oh, I gotcha -- in that sort of case it makes great sense.  In my case there would currently be 2,700+ entries in there, so it's a lot less feasible for me.

What about putting your dlls into the GAC?  That's just the sort of thing the GAC is supposed to be used for.

I already looked long and hard at the GAC, but if I'm understanding it correctly any new dll would have to be registered on the web server itself through IIS or a similar management console.  Doesn't it also have to be re-registered on a change for that change to take effect?  Not sure about that one.  Basically I want to be able to make a change to the source code and have the rebuild done and testing started in less than 10 seconds (preferably <5), and messing with the GAC doesn't sound like it'd be in that ballpark.

Or can I just drop new/changed dlls in the GAC folder and it picks them up like dlls dropped in the bin folder?

Oh, now I see what you are saying.  Yes, you'd have to re-register with the GAC each time, so this is not ideal for a dev machine, but for production it would work.  Here's something else I've done in the past: have multiple hierarchical references to folders that might contain your dll.  So in other words the GAC is at the bottom of the list (it always looks there if it doesn't find it elsewhere, I think), but then your specific reference comes from the other location that is hardcoded on your machine.  If multiple dev machines have different locations, you can put reference paths in for each of them.

I used to do that all the time with VS.NET (the original version from 2001 or 2002, whatever year that was).  However, in VS2005 and onward they changed the way the references work to some degree, and I don't know if those changes break the ability to do that sort of thing.

[keith.lamothe]

but then your specific reference comes from the other location that is hardcoded on your machine

I don't think I understand, are these reference paths some sort of file system symbolic link, or some sort of web.config option?

Thanks,
Keith

[x4000]

but then your specific reference comes from the other location that is hardcoded on your machine

I don't think I understand, are these reference paths some sort of file system symbolic link, or some sort of web.config option?

Thanks,
Keith

Well, the way it used to be was that each "reference" was actually something like this:

DLLA.dll
DLLB.dll
DLLC.dll

And then you'd set a list of reference paths:

C:\
C:\myproject
D:\shared\otherproject

And so it would go down the chain of reference paths, looking for a dll with each name.  As soon as it found it in one place it would stop looking, so there was a higher precedence for the top-level reference paths.  And it would also look in your application's bin folder always as the very first thing, and the GAC as the very last thing.  Then in 2005 they changed that all to a degree, and I never had a reason to look into it much to see if the above sort of thing was still possible.  But that's how it was at one time, I used to make good use of that (but by the time 2005 rolled around, I no longer was).

[keith.lamothe]

So you had a way to tell it to look in the C:\ for a dll?  From what I understand the current (3.5) system won't let you do that unless the home directory of the website is "C:\"; if it's "C:\inetpub\" it won't allow it.

[x4000]

So you had a way to tell it to look in the C:\ for a dll?  From what I understand the current (3.5) system won't let you do that unless the home directory of the website is "C:\"; if it's "C:\inetpub\" it won't allow it.

Hmm, at work we are still using .NET 2.0 actually, so I can't comment on .NET 3.5 ASP.NET.  But I'm surprised they would remove that ability so thoroughly.

Although, one thing that is relevant, is how IIS handles its shadow folders (as I think they were called).  Essentially, you drop a new dll/version in there, and then it has a file system watcher and copies that dll to the directory it really uses, restarting the web app at the same time.  So maybe if your dlls are elsewhere in the system, it can't handle that.

But, more to the point, I'm not actually suggesting that you use far references in a production application, just that if you have a reference like that in a dev application, and you've set "copy local" to true on that reference, it will copy it to the bin folder anyway, and IIS will be happy.

Or, the other tried and true method if your updated project is separate from the primary web project:  use a batch file on compile to copy it over to the right bin folder.  For example, in one of my secondary projects at work, in VS2008 under the Build Events tab, I just have added the following under Post-build event command line:

xcopy C:\Inetpub\wwwroot\StartaServices\bin\StartaServices.* C:\Inetpub\wwwroot\Jericho\bin\ /Y

xcopy C:\Inetpub\wwwroot\StartaServices\bin\Starta.Jericho* C:\Inetpub\wwwroot\Jericho\bin\ /Y

That ought to do it, if nothing else does.

[keith.lamothe]

Yea, our approach is to have an auto-copying service, since the machines doing the actual builds aren't necessarily connected to the webserver.  I just wanted to see if you had anything cleaner because we're going to potentially have 100 different bin folders for the wide variety of IIS websites on our server.  In the long run we may go with filesystem hard links so we get the actual concept of one authoritative location that's simply referenced.

[x4000]

Yea, our approach is to have an auto-copying service, since the machines doing the actual builds aren't necessarily connected to the webserver.  I just wanted to see if you had anything cleaner because we're going to potentially have 100 different bin folders for the wide variety of IIS websites on our server.  In the long run we may go with filesystem hard links so we get the actual concept of one authoritative location that's simply referenced.

Ah, gotcha.  Yeah, you might look into the reference path things, too, that might save you some grief.  Maybe not, but it's worth a look at least.