RegularExpressionSupport - Roundup Wiki here

RegularExpressionSupport

Immutable Page
Comments
Info
Attachments
More Actions:

Introduction

This document describes how to add regular expression search support to your

roundup tracker. The syntax for the regular expressions is of course the same syntax as used in Python's RE module. For more information about the syntax, we refer to the documentation belonging to that module.

Requirements

The DatabaseWrapper is required if you want to add the regular expression

search to your tracker.

Implementation

The regular expression feature consists out of a new search HTML template

and a new action handler that supports regular expressions.

_Search HTML Template_<br> The new template is based on the 'issue.search.html' template and it came out of the roundup 1.1.1 distribution. To see the changes, please run a diff against the new 'issue.search.html' template and the out of the box one from 1.1.1. Don't get scared. The changes are simple and not too much.

_Action Handler_<br> The action handler is also based on a standard out of the box action handler of roundup 1.1.1. That action handler is 'SearchAction' in 'actions.py'. The new handler supports all the features of the original handler as long as the user didn't ask for a regular expression search. To see the changes, please run a diff against the original action handler.

Best regards,<br> Marlon van den Berg

PS: Be aware that a regular expression search can consume some time when you run it on a slow server or in a tracker with a huge number of issues and messages.

<hr>

Source Code

Here is the new 'issue.search.html' template

<tal:block metal:use-macro="templates/page/macros/icing"> <title metal:fill-slot="head_title" i18n:translate="">Issue searching - <span
- i18n:name="tracker" tal:replace="config/TRACKER_NAME" /></title>
<span metal:fill-slot="body_title" tal:omit-tag="python:1"
- i18n:translate="">Issue searching</span>
<td class="content" metal:fill-slot="content">
<disabled script language="javascript" type="text/javascript">
- //
<disabled /script>
<form method="GET" name="itemSynopsis"
- tal:attributes="action request/classname">
<table class="form" tal:define="
- cols python:request.columns or 'id activity title status assignedto'.split(); sort_on python:request.sort[1] or 'activity'; group_on python:request.group[1] or 'priority'; search_input templates/page/macros/search_input; column_input templates/page/macros/column_input; sort_input templates/page/macros/sort_input; group_input templates/page/macros/group_input; search_select templates/page/macros/search_select;
  search_multiselect templates/page/macros/search_multiselect;">
<tr tal:define="name string:@search_text">
- <th i18n:translate="">All text*:</th> <td>
  - <input size="45"
    - tal:attributes="value python:request.form.getvalue(name) or nothing;
      - name name">
  </td> <td nowrap colspan="3">
  - <input type="checkbox" name="reg_exp" value="1" onClick="javascript: onClick_RegExp()"
    - tal:attributes="checked reguest/form/reg_exp/value | nothing">
    Regular Expression<br> <input type="checkbox" name="re_ignorecase" value="1"
    - tal:attributes="checked reguest/form/re_ignorecase/value | nothing">
    Ignore case<br> <br>
  </td>
</tr>
<tr>
- <th class="header"> </th> <th class="header" i18n:translate="">Filter on</th> <th class="header" i18n:translate="">Display</th> <th class="header" i18n:translate="">Sort on</th> <th class="header" i18n:translate="">Group on</th>
</tr>
<tr tal:define="name string:title">
- <th i18n:translate="">Title:</th> <td metal:use-macro="search_input"></td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td> </td>
</tr>
<tr tal:define="name string:topic;
- db_klass string:keyword;
  db_content string:name;">
- <th i18n:translate="">Topic:</th> <td metal:use-macro="search_select"></td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:id">
- <th i18n:translate="">ID:</th> <td metal:use-macro="search_input"></td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td> </td>
</tr>
<tr tal:define="name string:creation">
- <th i18n:translate="">Creation Date:</th> <td metal:use-macro="search_input"></td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:creator;
- db_klass string:user; db_content string:username;"
- tal:condition="db/user/is_view_ok">
- <th i18n:translate="">Creator:</th> <td metal:use-macro="search_select">
  - <option metal:fill-slot="extra_options" i18n:translate=""
    - tal:attributes="value request/user/id">created by me</option>
  </td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:activity">
- <th i18n:translate="">Activity:</th> <td metal:use-macro="search_input"></td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td> </td>
</tr>
<tr tal:define="name string:actor;
- db_klass string:user; db_content string:username;"
- tal:condition="db/user/is_view_ok">
- <th i18n:translate="">Actor:</th> <td metal:use-macro="search_select">
  - <option metal:fill-slot="extra_options" i18n:translate=""
    - tal:attributes="value request/user/id">done by me</option>
  </td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td> </td>
</tr>
<tr tal:define="name string:priority;
- db_klass string:priority;
  db_content string:name;">
- <th i18n:translate="">Priority:</th> <td metal:use-macro="search_select">
  - <option metal:fill-slot="extra_options" value="-1" i18n:translate=""
    - tal:attributes="selected python:value == '-1'">not selected</option>
  </td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:status;
- db_klass string:status;
  db_content string:name;">
- <th i18n:translate="">Status:</th> <td metal:use-macro="search_select">
  - <tal:block metal:fill-slot="extra_options">
    - <option value="-1,1,2,3,4,5,6,7" i18n:translate=""
      - tal:attributes="selected python:value == '-1,1,2,3,4,5,6,7'">not resolved</option>
      <option value="-1" i18n:translate=""
      - tal:attributes="selected python:value == '-1'">not selected</option>
    </tal:block>
  </td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:assignedto;
- db_klass string:user; db_content string:username;"
- tal:condition="db/user/is_view_ok">
- <th i18n:translate="">Assigned to:</th> <td metal:use-macro="search_select">
  - <tal:block metal:fill-slot="extra_options">
    - <option tal:attributes="value request/user/id"
      - i18n:translate="">assigned to me</option>
      <option value="-1" tal:attributes="selected python:value == '-1'"
      - i18n:translate="">unassigned</option>
    </tal:block>
  </td> <td metal:use-macro="column_input"></td> <td metal:use-macro="sort_input"></td> <td metal:use-macro="group_input"></td>
</tr>
<tr>
- <th i18n:translate="">No Sort or group:</th> <td> </td> <td> </td> <td><input type="radio" name="@sort" value=""></td> <td><input type="radio" name="@group" value=""></td>
</tr>
<tr> <th i18n:translate="">Pagesize:</th> <td><input name="@pagesize" size="3" value="50"
- tal:attributes="value request/form/@pagesize/value | default"></td>
</tr>
<tr> <th i18n:translate="">Start With:</th> <td><input name="@startwith" size="3" value="0"
- tal:attributes="value request/form/@startwith/value | default"></td>
</tr>
<tr> <th i18n:translate="">Sort Descending:</th> <td><input type="checkbox" name="@sortdir"
- tal:attributes="checked python:request.sort[0] == '-' or request.sort[0] is None">
</td> </tr>
<tr> <th i18n:translate="">Group Descending:</th> <td><input type="checkbox" name="@groupdir"
- tal:attributes="checked python:request.group[0] == '-'">
</td> </tr>
<tr tal:condition="python:request.user.hasPermission('Edit', 'query')">
- <th i18n:translate="">Query name**:</th> <td tal:define="value request/form/@queryname/value | nothing">
  - <input name="@queryname" tal:attributes="value value"> <input type="hidden" name="@old-queryname" tal:attributes="value value">
  </td>
</tr>
<tr>
- <td>
  - <input type="hidden" name="@action" value="regexp_search">
  </td> <td><input type="submit" value="Search" i18n:attributes="value"></td>
</tr>
<tr><td> </td>
- <td colspan="4" class="help" i18n:translate="">
  - : The "all text" field will look in message bodies and issue titles<br> <span tal:condition="python:request.user.hasPermission('Edit', 'query')">
  - *: If you supply a name, the query will be saved off and available as a
    - link in the sidebar
    </span>
  </td>
</tr> </table>
</form>
<disabled script language="javascript" type="text/javascript">
- //
<disabled /script>
</td>
</tal:block>

And here is the new action handler source code::

import cgi, types # templating import required for roundup 1.3.2, at a minimum # from roundup.cgi import templating
from roundup.cgi.actions import SearchAction from roundup.cgi.exceptions import Redirect from roundup.extensions import dbwrapper
class RegExpSearchAction(SearchAction):
- def handle(self):
  - """\ Mangle some of the form variables. Set the form ":filter" variable based on the values of the filter variables - if they're set to anything other than "dontcare" then add them to :filter. Handle the ":queryname" variable and save off the query to the user's query list. Split any String query values on whitespace and comma. """ self.fakeFilterVars() queryname = self.getQueryName() # Prepare for regular expression if self.form.has_key('reg_exp'):
    - self.form.value.remove(self.form['reg_exp']) if queryname:
      - error_message = Sorry, regular expression searches can't be saved yet self.client.error_message.append(error_message) self.client.template = 'search' return
      elif not self.form.has_key('@search_text'):
      - error_message = Missing regular expression in 'All text' field self.client.error_message.append(error_message) self.client.template = 'search' return
      else:
      - reg_exp = self.form['@search_text'].value
      if self.form.has_key('re_ignorecase'):
      - re_ignorecase = int(self.form['re_ignorecase'].value)
      else:
      - re_ignorecase = 0
    else:
    - reg_exp = None
    # editing existing query name?
    old_queryname = for key in ('@old-queryname', ':old-queryname'):
    - if self.form.has_key(key):
      old_queryname = self.form[key].value.strip()
    # handle saving the query params if queryname:
    - # parse the environment and figure what the query _is_ req = templating.HTMLRequest(self.client) # The [1:] strips off the '?' character, it isn't part of the # query string.
      url = req.indexargs_url(, {})[1:] key = self.db.query.getkey() if key:
      # edit the old way, only one query per name try:
      qid = self.db.query.lookup(old_queryname) if not self.hasPermission('Edit', 'query', itemid=qid):
      raise exceptions.Unauthorised, self._(
      "You do not have permission to edit queries")
      self.db.query.set(qid, klass=self.classname, url=url)
      except KeyError:
      # create a query if not self.hasPermission('Create', 'query'):
      raise exceptions.Unauthorised, self._(
      "You do not have permission to store queries")
      qid = self.db.query.create(name=queryname,
      klass=self.classname, url=url)
      else:
      # edit the new way, query name not a key any more # see if we match an existing private query uid = self.db.getuid() qids = self.db.query.filter(None, {'name': old_queryname,
      'private_for': uid})
      if not qids:
      # ok, so there's not a private query for the current user # - see if there's one created by them qids = self.db.query.filter(None, {'name': old_queryname,
      'creator': uid})
      if qids:
      # edit query - make sure we get an exact match on the name for qid in qids:
      if old_queryname != self.db.query.get(qid, 'name'):
      continue
      if not self.hasPermission('Edit', 'query', itemid=qid):
      raise exceptions.Unauthorised, self._( "You do not have permission to edit queries")
      self.db.query.set(qid, klass=self.classname,
      url=url, name=queryname)
      else:
      # create a query if not self.hasPermission('Create', 'query'):
      raise exceptions.Unauthorised, self._(
      "You do not have permission to store queries")
      qid = self.db.query.create(name=queryname,
      klass=self.classname, url=url, private_for=uid)
      # and add it to the user's query multilink queries = self.db.user.get(self.userid, 'queries') if qid not in queries:
      queries.append(qid) self.db.user.set(self.userid, queries=queries)
      # commit the query change to the database self.db.commit()
    # check for regular expression search if reg_exp and self.classname == 'issue':
    - # not needed anymore if self.form.has_key('@search_text'):
      - self.form.value.remove(self.form['@search_text'])
      # open dbwrapper dbw = dbwrapper.DBWrapper(self.db) if re_ignorecase:
      - flags = dbwrapper.IGNORECASE
      else:
      - flags = 0
      flags += dbwrapper.LOCALE issues = [ issue.id for issue, res in dbw.issue.re.search(reg_exp, flags, 'title') ] flags += (dbwrapper.MULTILINE + dbwrapper.DOTALL) messages = dbw.msg.re.search(reg_exp, flags, 'content') results = dbw.issue.find(messages=messages) for issue in results:
      - if not issue.id in issues:
        issues.append(issue.id)
      if self.form.has_key('id'):
      - if isinstance(self.form['id'], types.ListType):
        form_issues = [] for minifield in self.form['id']:
        if minifield.value:
        form_issues += re.split('[+, ]', minifield.value)
        else:
        form_issues = re.split('[+, ]', self.form['id'].value)
        self.form.value.remove(self.form['id']) found_issues = [] for issue_id in form_issues:
        if int(issue_id) in issues:
        found_issues.append(int(issue_id))
        issues = found_issues
      if issues:
      - issues.sort()
      else:
      - issues.append(-1)
      self.form.value.append(cgi.MiniFieldStorage('id', '+'.join( [str(nodeid) for nodeid in issues] ))) args = [] for column in self.form.keys():
      - if isinstance(self.form[column], types.ListType):
        values = [] for minifield in self.form[column]:
        if minifield.value:
        values.append(minifield.value)
        else:
        values = [self.form[column].value]
        args.append('%s=%s'%(column, ','.join(values)))
      raise Redirect, '%s%s?&%s'%(self.base, self.classname, '&'.join(args))
def init(instance):
- instance.registerAction('regexp_search', RegExpSearchAction)