Recess Developer Forums: Complex Boolean SQL Queries - Recess Developer Forums

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

Complex Boolean SQL Queries OR's and AND's should be more readily possible!

#1 User is offline   KrisJordan Icon

  • Administrator
  • Icon
  • Group: Administrators
  • Posts: 74
  • Joined: 25-August 09
  • LocationNorth Carolina, USA

Posted 17 September 2009 - 04:47 PM

Michelle brings up a point of improvement that has been needed for some time now. OR support.

View Postmichelle, on 17 September 2009 - 05:17 PM, said:

Is is possible to search multiple columns in a table for the same pattern using the mysql 'OR' keyword?
Something that would result in a statement similar to:

SELECT * FROM posts WHERE title LIKE '%keyword%' OR description LIKE '%keyword%';

Thanks!


There was once some discussion on how this could be implemented with chaining still possible, so, it would look something like:

$post->or(Cond::equal('title','%keyword%'), Cond::equal('description','%keyword%'))->orderBy('date DESC');

Who is itching to get their hands dirty with SQLBuilder?
0

#2 User is offline   christiaan Icon

  • Group: Members
  • Posts: 33
  • Joined: 06-September 09
  • LocationNijkerk, Netherlands

Posted 18 September 2009 - 01:22 AM

I've actually hit this wall also yesterday. Have been able to work around it thus far tough.

I think to keep things consistent if we decide to go this way
$post->or(Cond::equal('title','%keyword%'), Cond::equal('description','%keyword%'))->orderBy('date DESC');


The whole normal where shizzle should be changed to support proper nesting for AND as well
Imagine the following query:
SELECT * FROM posts
WHERE posted BETWEEN '2 weeks ago' AND 'today'
AND (
(title LIKE '%keyword%' AND active = 1) OR
title LIKE '%keyword%')
ORDER BY date DESC;

Resulting in a syntax like
$post->where(Cond::and(Cond::or(Cond::and(Cond::like('title','%keyword%'), Cond::equal('active', true)),
Cond::like('description','%keyword%')), Cond::between('posted', '-2 weeks', 'today'))->orderBy('date DESC');

Which I find really, really cumbersome. I like ease with which you can write the queries now.

I personally tought about the following syntax yesterday (I have really no idea if this would be possible with the current internal workings)
$post->between('posted', '-2 weeks', 'today')->
or($post->like('title', '%keyword%')->equal('active', true), $post->like('title', '%keyword%'))->orderBy('date DESC');

0

#3 User is offline   KrisJordan Icon

  • Administrator
  • Icon
  • Group: Administrators
  • Posts: 74
  • Joined: 25-August 09
  • LocationNorth Carolina, USA

Posted 18 September 2009 - 06:48 PM

View Postchristiaan, on 18 September 2009 - 02:22 AM, said:

I've actually hit this wall also yesterday. Have been able to work around it thus far tough.


Yes, this is a high priority. I am going to start moving on it soon.

View Postchristiaan, on 18 September 2009 - 02:22 AM, said:

I think to keep things consistent if we decide to go this way
$post->or(Cond::equal('title','%keyword%'), Cond::equal('description','%keyword%'))->orderBy('date DESC');


The whole normal where shizzle should be changed to support proper nesting for AND as well


The way it currently works every condition is and'ed by default. We could have two methods for flipping that switch, either andAll()/orAll(). andAll() is the default:

$post->
   orAll()->
   like('title','%keyword%')->
   like('description','%keyword%')->
   orderBy('date DESC');


This addresses simple cases but would lead to weird cases when trying to do queries over relationships:

$authorsOfKeywordWithName = 
$post->
   orAll()->
   like('title','%keyword%')->
   like('description','%keyword%')->
   author()->
     like('firstname','%name%')->
     like('lastname','%name%');
[/quote]

Would the author params be or'ed to? What would you expect?

[quote name='christiaan' date='18 September 2009 - 02:22 AM' timestamp='1253254952' post='195']
Resulting in a syntax like
[php]
$post->where(Cond::and(Cond::or(Cond::and(Cond::like('title','%keyword%'), Cond::equal('active', true)),
Cond::like('description','%keyword%')), Cond::between('posted', '-2 weeks', 'today'))->orderBy('date DESC');
[/php]
Which I find really, really cumbersome. I like ease with which you can write the queries now.
[/quote]

Agreed, this is ugly.

[quote name='christiaan' date='18 September 2009 - 02:22 AM' timestamp='1253254952' post='195']
I personally tought about the following syntax yesterday (I have really no idea if this would be possible with the current internal workings)
[php]
$post->between('posted', '-2 weeks', 'today')->
or($post->like('title', '%keyword%')->equal('active', true), $post->like('title', '%keyword%'))->orderBy('date DESC');
[/php]
[/quote]

Unfortunately you don't always have a reference to the object you're making these queries on (two specific cases: Make::a('Post')->betwee... and when you're specifying constraints on a related / joined object/table).

[quote name='Rafał Filipek' date='18 September 2009 - 07:06 PM' timestamp='1253315213' post='200']
I was thinking about all those complex find queries etc. I don't like it. As I can see You trying to keep this framework easy to use. When I'm looking at Your example Kris I can't say it is easy. IMHO 
[code]$post->or(Cond::equal('title','%keyword%'), Cond::equal('description','%keyword%'))->orderBy('date DESC');
is much more complicated than
$post->where('title LIKE "?" OR description LIKE "?"', 'foo','bar')->orderBy('date DESC');

Your example looks good for simple queries. But what about
{condition1} or {{condition2} and {condition3}}
etc.
It would be damn hard to implement so complex subconditions etc IMO.
I think we could look at Doctrine in this case : http://www.doctrine-...#select-queries
[/quote]

Having a mini-sql language is a good idea, it'll take some thought to get right though and will have to be driven by a simpler programmatic API underneath the covers.

Let's keep thinking on this.
0

#4 User is offline   Rafał Filipek Icon

  • Group: Members
  • Posts: 35
  • Joined: 28-August 09
  • LocationPoland

Posted 18 September 2009 - 07:14 PM

Ok, another idea :) Why don't we merge some simple mini-sql language ( we should keep it realy simple ) and Cond::stuff(). Cond::stuff() can be very usefull in some cases, like
Cond::in('id', array(1,4,7));
Cond::like('title', '%foo%');

etc. In those examples Cond::stuff is building some quite long string with is a part of sql query. For sure it more usefull than
$post->where('id IN(?,?,?)', 1, 4, 7);

So in fact we can do smth like.
$post->where('id? or title?', Cond::in(array(1,4,7)), Cond::like('%foo%'));

Cond::sutff don't has to know the name of the column. All it has to do is genereate some string like
Cond::in(array(1, 4, 7)); # [space]IN (1,4,7)[space]
Cond::like('%foo%'); # [space]LIKE '%foo%'[space]

that's all.
? sign is showing us where we have to put each string generated by Cond method. If we want to do smth with column name ( add ad table name ) we just have to grab by some simple RegExp a-/zA-Z0-9_/ string and the ? sign, do smth with column name and replace ? with sql part.
So now we could keep structure of compex where statements simple and use Cond::stuff etc to generate more complex parts of query
$post->where('id? and title? or (id? or (description? and foo?)))', Cond::equal(3), Cond::like('%Rafal%'), Cond::in(1,4,7), Cond::like('Filipek%'), Cond::not('bar'));

[ColumnName]? - maybe it's not the best syntax :)
Yes, in Poland we have a internet connection ;]
0

#5 User is offline   Thomaz Icon

  • Group: Members
  • Posts: 30
  • Joined: 28-August 09
  • LocationHeemskerk, The Netherlands

Posted 20 September 2009 - 08:02 AM

View PostRafał Filipek, on 19 September 2009 - 02:14 AM, said:

$post->where('id? and title? or (id? or (description? and foo?)))', Cond::equal(3), Cond::like('%Rafal%'), Cond::in(1,4,7), Cond::like('Filipek%'), Cond::not('bar'));


Well I don't think this is a very good solution, because complex queries are always quite lengthy so I think that you will loose track of all the conditions in such a query; like which Cond:: belongs to the third "?"?

Something like

$post->where('id{id} and title{title} or (id{id2} or (description{desc} and foo{foo})))', array("id"=>Cond::equal(3), "title"=>Cond::like('%Rafal%'), "id2"=>Cond::in(1,4,7), "desc"=>Cond::like('Filipek%'), "foo"=>Cond::not('bar')));


allows you to skim the query faster and keep track of conditions easier, but it has its drawbacks too..
Hoppa
0

#6 User is offline   Rafał Filipek Icon

  • Group: Members
  • Posts: 35
  • Joined: 28-August 09
  • LocationPoland

Posted 20 September 2009 - 09:48 AM

Hmmm, maybe you're right. There is only one element in your example, which I don't like. {id}, {id2}. If we want to identify each Cond in query imo we should do this like that:
$post->where(
	'{id} and {title} or ({id} or({description} and {title}))', 
	array(
		'id' => array(Cond::equal(3), Cond::in(1,4,7)),
		'title' => array(Cond::like('%Rafal%'), Cond::like('%Filipek%'),
		'description' => Cond::like('%Recess Framework%')
	);
);

Now we have some logical groups of conditions and it's easy to find specified Cond.
Yes, in Poland we have a internet connection ;]
0

#7 User is offline   KrisJordan Icon

  • Administrator
  • Icon
  • Group: Administrators
  • Posts: 74
  • Joined: 25-August 09
  • LocationNorth Carolina, USA

Posted 22 September 2009 - 11:22 AM

In learning more about jQuery this weekend I found out about how jQuery handles a similar problem with the end() function. Let me demo:

$('#myBox')
    .children('.myClass') 
       .hide()
    .end()
    .blink();


The end function returns the focus back to $("#myBox") after it was 'nested' in a more complex expression. A similar concept expressed in PHP/Recess would look like:

<?php
$post->
   or()->
     like('title','%keyword%')->
     like('description','%keyword%')->
   end()->   
   orderBy('date DESC');


If we were only concerned about one level of conditional nesting we could instead place the boolean methods between statements with no end(), or as prefixes of each selector:

<?php
$post->
   like('title','%keyword%')->
   or()->
   like('description','%keyword%')->
   orderBy('date DESC');


Or

<?php
$post->
   like('title','%keyword%')->
   orLike('description','%keyword%')->
   orderBy('date DESC');


From a readability point of view I'm a fan of the second to last option just mentioned. It's a simple solution that reads well. Perhaps nesting (which I believe is a rare case and not the case to optimize for) could be accomplished with additional syntax that's a little uglier:

<?php
$post->
   nest()->
     like('title','%keyword%')->
     or()->
     like('description','%keyword%')->
   endNest()->
   and()->
   nest()->
     lessThan('createdAt',123456)->
     or()->
     greaterThan('description','%keyword%')->
   endNest()->
   orderBy('date DESC');


When no and/or is specified the default behavior is an and() (how it currently works).

<?php
$post->like('description','%keyword%')->or()->like('title','%keyword%')->lessThan('createdAt',12345);
// Equivalent to
$post->like('description','%keyword%')->or()->like('title','%keyword%')->and()->lessThan('createdAt',12345);
// OR on all params:
$post->like('description','%keyword%')->or()->like('title','%keyword%')->or()->lessThan('createdAt',12345);
/// The following are the same:
$post->like('description','%keyword%')->like('title','%keyword%')->lessThan('createdAt',12345);
$post->like('description','%keyword%')->and()->like('title','%keyword%')->and()->lessThan('createdAt',12345);


The readability of this is growing on me.
0

#8 User is offline   Rafał Filipek Icon

  • Group: Members
  • Posts: 35
  • Joined: 28-August 09
  • LocationPoland

Posted 25 September 2009 - 09:43 AM

I like Your soluition Kris. But i think it would be hard to implement if You will have nested nests, like:
$post->
   nest()->
     like('title','%keyword%')->
     or()->
     like('description','%keyword%')->
     or()->
          nest()->
               lessThan('createdAt',123456)->
               or()->
               greaterThan('description','%keyword%')->
         endNest()->
   endNest()->
   orderBy('date DESC');

I was working on similar solution one day but i couldn't implement it :)
Yes, in Poland we have a internet connection ;]
0

#9 User is offline   spronkey Icon

  • Group: Members
  • Posts: 1
  • Joined: 16-November 09

Posted 16 November 2009 - 06:00 PM

Hi all - new to these parts, but very interested in this framework - excellent job so far guys, and with the right philosophy in my opinion!

Forgive the fact that I'm not yet 100% up to speed on the internals (working on it ;)), but I recently spent some time at my job working on our own internal framework's query and forms APIs. We quickly discovered that not only did we simply need complex boolean logic for queries, but we also needed to take further advantage of SQL when we needed to squeeze performance using switch cases, functions etc - so we basically abstracted SQL into objects. I don't know whether you folks think going this way is overkill or not, but I think it's definitely worth keeping in consideration when designing the query API. Our object model for query abstraction was similar to the following:

// top-level interfaces and AliasableExpression abstract class
interface Query {} // represents query objects as they are being built
interface IExpression {} // represents SQL expressions
interface ISelectableExpression {} // represents those that can be used in SELECT fields
interface IGroupable {} // represents groupable properties or expressions
class Alias implements IExpression, ISelectableExpression {} // represents AS with decorator pattern
abstract class AliasableExpression implements IExpression {} // represents expressions that can be aliased with AS

/* individual sql expression types
  * - SimpleValue e.g. 1, 'woot', 33.502 etc
  * - Column e.g. table.column
  * - Function e.g. REPLACE(arg,arg,arg,arg)
  * - Evaluation e.g. AliasableExpression {OPERATOR} AliasableExpression e.g. table.column = REPLACE(x,y,z)
  * - ExpressionSet with operator e.g. AliasableExpression AND AliasableExpression AND AliasableExpression
  * - Cast e.g. CAST(AliasableExpression, DATETIME)
  * - Case e.g. CASE WHEN ... THEN ... ELSE ... END
  */
class SimpleValue extends AliasableExpression implements ISelectableExpression {
   ::factory(Query $q, $value, $type);
}
class Column extends AliasableExpression implements ISelectableExpression, IGroupable {
  ::factory(Query $q, $columnOrAlias, $table = null);
}
class Function extends AliasableExpression {
  ::factory(Query $q, $functionName);
  addExpression(AliasableExpression $e);; // arguments in order - arguments can be any aliasable expression, even a switch case
}
class Evaluation extends AliasableExpression {
  ::factory(Query $q, AliasableExpression $this, $operator, AliasableExpression $that);
}
class ExpressionSet extends AliasableExpression {
  ::factory(Query $q, $logical_operator);
  addExpression(AliasableExpression $e);; // expressions imploded e.g. AE[0] $logical_operator AE[1] $logical_operator AE[2] $logical_operator AE[3]
}

/* alias class - can be used when selecting but not in other locations */
class Alias implements IExpression, ISelectableExpression {
  ::factory(IExpression $e, $alias); // $e AS $alias
}


We never got around to building the fluent query API, but we did build a (sort-of) fluent forms API using objects similar to those above. Our challenge was that we always needed access to the $form object, and had to deal with arbitrary numbers of nested containters, so a true fluent API was too serialised for our needs. The end-result we came up with produces streams as follows:

$model = new ModelObject();
$form = new Form(new URI());
$form->build(
  $form->container(
    $form->container(
      new FormModelBuilder(array('name', 'ipaddress', 'creationdate'), $model),
      new WidgetNullStateMessage(Lang::message(CREATE_A_MACHINE)
    )->template('some_form_template.tpl'),
    new FormModelBuilder(array('name2', 'someotherfield', 'creationdate'), $model)
  )
);


Reworking this into a SQL query API you can see how it could work out as something along the following lines. I'm not 100% sure I agree with you Kris regarding nesting being a rare case - I find that we often need to nest especially where using complex data views with user filtering. But I'm not sure whether this can be handled automatically outside of customised queries.

// in the following example, $post->query() uses func_get_args() to allow multiple arguments, where the and(), or(), like(), and gt() methods each return an expression
$filter = $post->filter();
$post->query(
  $post->and(
    $filter->or(
      $filter->like('title', '%keyword%'), // returns a new Like object and adds it to the set
      $filter->like('description', '%keyword%') 
    ), // returns the set with two like objects and OR operator/concatenator
    $filter->and( // the and is redundant here
      $filter->gt('creationDate', DATE_NOW)
    ), // returns the set with one filter
    $filter->gt('modificationDate', DATE_YESTERDAY), // works outside a set.
  )
)->orderBy(
   'property DESC'
); // returns the filter AND concatenator and two nested sets, and a filter applied

// outputs something along these lines
// SELECT * FROM posts WHERE ((title LIKE %keywords% OR description LIKE %keywords%) AND (creationDate > '2009-11-16 12:02:33') AND modificationDate > '2009-11-15 00:00:00'


It's not quite a 'fluent' API, but it's similar. It's effectively using a basic factory pattern on the $filter object with stuff like like(), gt(). In our framework we simply used Expression() and passed in an operator, but that is a little ugly as you have to pull the operator from somewhere.

Anyhow, I thought this might come in handy! Thoughts?
0

#10 User is offline   cuhuak Icon

  • Group: Members
  • Posts: 1
  • Joined: 17-December 09

Posted 17 December 2009 - 03:04 AM

Hi all!

I like Kris solution too.
Also I want to suggest my solution -
it is mix of propel style (using criteria) and doctrine style (using chaining) + Kris code:

$c1 = criteria($post)->
     like('title','%keyword%')->
     or()->
     like('description','%keyword%');

$c2 = criteria($post)->
     lt('createdAt',123456)->
     or()->
     like('description','%keyword%');

$post->
    criteria($c1)->
    and()->
    criteria($c2);


And what do you think about this?
0

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users