2010年2月17日星期三

Re: [fw-db] Caching SQL query results

On Tue, Feb 16, 2010 at 10:18 AM, Nicolas Grevet <ngrevet@alteo.fr> wrote:
> I never quite understood how you're supposed to handle page caching
> expiration mechanisms. With a database caching, this is 'somewhat' easy
> (empty the cache on insert, update and delete events) but on a page? How do
> you declare a page obsolete as soon as something used inside it change
> inside the database?

Often you dont, you just rely on time : the page expires after X
seconds of time, X can be threshold with the application load.
Not talking about HTTP cache mechanisms which can help as well.

>
> We hooked a memcached server right before the execution of the Zend_Db
> component in order to catch the request before they're sent to the server,
> it allows us to save hundreds of requests, but why in the first place is
> there more than hundreds of requests per page anyway? I guess this is the
> problem with every ORM, but I'm not really sure.

That's a good point, all Zend_Db components queries goes throught
Adapter::query(), that's where to work (with memcache)
Zend_Db is not an ORM, ORMs often have full caching features that are
even more complex that a query hash based cache, see Doctrine's one,
or Doctrine2's one (or Hibernate's ones)

>
> Of course we have to extract the content of 35 tables in order to build our
> (very) complex page, that may be the problem, but then how are you supposed
> to handle page-level caching when any information of these 35 tables can
> change at any moment and modify the page? Memcaching SQL queries allowed us
> to save probably more than 500 or 600% of the loading time, but that's still
> not quite the best solution out there.
>

If your actual solution seems to make the job, leave it as is, if not
: perhaps you should consider reviewing your DB model, create DB views
or trigger, or even try to migrate to some nosql DB ?


J.P


> - Nicolas
>
>
> Abraham Block wrote:
>>
>> Much of this can probably be done with page level caching, instead of
>> caching the SQL Queries. MySQL can cache queries itself, as well.
>>
>> On Thu, Jan 28, 2010 at 12:01 PM, Andreas Möller <localheinz@l8m.de
>> <mailto:localheinz@l8m.de>> wrote:
>>
>>        * In an e-commerce context, how many SQL queries would you
>>        consider to
>>        be too much, for example on a complex product page, with nested
>>        categories, products and multiple references, collections of
>>        products,
>>        multi-product kits, temporary events, promotions, and everything
>>        plus a
>>        bit of AJAX all over it?
>>
>>
>>    You mean, in a year?
>>
>>    Doesn't make any sense counting queries, if the total is not set in
>>    relation with a specified time.
>>
>>
>>        * We have a lot of these SQL queries at the moment, and we're
>>        considering enabling a SQL query result caching mecanism soon.
>>        But right
>>        now, we're wondering where we're supposed to add this kind of
>>        system in
>>        Zend_Db. There don't seem to be any useful hook around it (except
>>        insert, update & delete). We didn't find where we should hook
>>        the system
>>        because there's so much methods... find(), fetchRow(), fetchAll(),
>>        findDependentRowset(), findManyToManyRowset() and so on...
>> Moreover,
>>        since it's not designed to handle ORDB, there don't seem to be
>>        any way
>>        of catching IDs (or whatever column is used as an identifier) in
>> the
>>        queries. Which is kind of a problem if we want to use it as the
>>        cache
>>        identifier.
>>
>>
>>    Use
>>
>>     [x] Zend_Cache_Manager
>>
>>    and prepare as many cache templates as you like, and then cache
>>    results with cache instances retrieved from there.
>>
>>
>>        * On the other hand, we do know that MySQL already have some
>>        internal
>>        caching mecanism, but the application still sends dozens of
>>        duplicated
>>        requests to the server on the same page. We don't know if this
>>        cache is
>>        of any help or it's just smoke and mirrors.
>>
>>
>>    Maybe it helps taking a look at an ORM such as Doctrine?
>>
>>
>>        I already crawled the web, searching for elegant solutions, but
>>        all I
>>        found was some tips'n tricks that are of no use in our situation
>>        (static
>>        caching and manual caching at each query). We really need to
>>        automate
>>        the process because we're kinda building a massive application
>> here.
>>
>>
>>    How about caching at model level?
>>
>>
>>    Best regards,
>>
>>    Andreas
>>
>>
>

没有评论: