Back to index

moin  1.9.0~rc2
Classes | Functions | Variables
MoinMoin.mail.mailimport Namespace Reference

Classes

class  attachment
class  ProcessingError

Functions

def log
def decode_2044
def email_to_markup
def get_addrs
def process_message
def get_pagename_content
def import_mail_from_string
def import_mail_from_file
def import_mail_from_message

Variables

 infile = sys.stdin
 debug = False
tuple re_sigstrip = re.compile("\r?\n-- \r?\n.*$", re.S)
list request_url = sys.argv[1]
tuple request = ScriptContext(url=request_url)

Detailed Description

    MoinMoin - E-Mail Import into wiki

    Just call this script with the URL of the wiki as a single argument
    and feed the mail into stdin.

    @copyright: 2006 MoinMoin:AlexanderSchremmer,
        2006 MoinMoin:ThomasWaldmann
    @license: GNU GPL, see COPYING for details.

Class Documentation

class MoinMoin::mail::mailimport::ProcessingError

Definition at line 40 of file mailimport.py.


Function Documentation

Decodes header field. See RFC 2044. 

Definition at line 47 of file mailimport.py.

00047 
00048 def decode_2044(header):
00049     """ Decodes header field. See RFC 2044. """
00050     chunks = decode_header(header)
00051     chunks_decoded = []
00052     for i in chunks:
00053         chunks_decoded.append(i[0].decode(i[1] or 'ascii'))
00054     return u''.join(chunks_decoded).strip()

Here is the call graph for this function:

Here is the caller graph for this function:

def MoinMoin.mail.mailimport.email_to_markup (   request,
  email 
)
transform the (realname, mailaddr) tuple we get in email argument to
    some string usable as wiki markup, that represents that person (either
    HomePage link for a wiki user, or just the realname of the person). 

Definition at line 55 of file mailimport.py.

00055 
00056 def email_to_markup(request, email):
00057     """ transform the (realname, mailaddr) tuple we get in email argument to
00058         some string usable as wiki markup, that represents that person (either
00059         HomePage link for a wiki user, or just the realname of the person). """
00060     realname, mailaddr = email
00061     u = user.get_by_email_address(request, mailaddr)
00062     if u:
00063         markup = u.wikiHomeLink()
00064     else:
00065         markup = realname or mailaddr
00066     return markup

Here is the caller graph for this function:

def MoinMoin.mail.mailimport.get_addrs (   message,
  header 
)
get a list of tuples (realname, mailaddr) from the specified header 

Definition at line 67 of file mailimport.py.

00067 
00068 def get_addrs(message, header):
00069     """ get a list of tuples (realname, mailaddr) from the specified header """
00070     dec_hdr = [decode_2044(hdr) for hdr in message.get_all(header, [])]
00071     return getaddresses(dec_hdr)

Here is the call graph for this function:

Here is the caller graph for this function:

def MoinMoin.mail.mailimport.get_pagename_content (   request,
  msg 
)
Generates pagename and content according to the specification
    that can be found on MoinMoin:FeatureRequests/WikiEmailintegration 

Definition at line 124 of file mailimport.py.

00124 
00125 def get_pagename_content(request, msg):
00126     """ Generates pagename and content according to the specification
00127         that can be found on MoinMoin:FeatureRequests/WikiEmailintegration """
00128     generate_summary = False
00129     choose_html = True
00130 
00131     cfg = request.cfg
00132     email_subpage_template = cfg.mail_import_subpage_template
00133     email_pagename_envelope = cfg.mail_import_pagename_envelope
00134     wiki_addrs = cfg.mail_import_wiki_addrs
00135     search_list = cfg.mail_import_pagename_search
00136     re_subject = re.compile(cfg.mail_import_pagename_regex)
00137 
00138     subj = msg['subject'].strip()
00139     pagename_tpl = ""
00140     for method in search_list:
00141         if method == 'to':
00142             for addr in msg['target_addrs']:
00143                 if addr[1].strip().lower() in wiki_addrs:
00144                     pagename_tpl = addr[0]
00145                     # special fix for outlook users :-)
00146                     if pagename_tpl and pagename_tpl[-1] == pagename_tpl[0] == "'":
00147                         pagename_tpl = pagename_tpl[1:-1]
00148                     if pagename_tpl:
00149                         break
00150         elif method == 'subject':
00151             m = re_subject.search(subj)
00152             if m:
00153                 pagename_tpl = m.group(1)
00154                 # remove the pagename template from the subject:
00155                 subj = re_subject.sub('', subj, 1).strip()
00156         if pagename_tpl:
00157             break
00158 
00159     pagename_tpl = pagename_tpl.strip()
00160     # last resort
00161     if not pagename_tpl:
00162         pagename_tpl = email_subpage_template
00163 
00164     if not subj:
00165         subj = '(...)' # we need non-empty subject
00166     msg['subject'] = subj
00167 
00168     # for normal use, email_pagename_envelope is just u"%s" - so nothing changes.
00169     # for special use, you can use u"+ %s/" - so you don't need to enter "+"
00170     # and "/" in every email, but you get the result as if you did.
00171     pagename_tpl = email_pagename_envelope % pagename_tpl
00172 
00173     if pagename_tpl.endswith("/"):
00174         pagename_tpl += email_subpage_template
00175 
00176     subject = msg['subject'].replace('/', '\\') # we can't use / in pagenames
00177 
00178     # rewrite using string.formatter when python 2.4 is mandatory
00179     pagename = (pagename_tpl.replace("$from", msg['from_addr'][0]).
00180                 replace("$date", msg['date']).
00181                 replace("$subject", subject))
00182 
00183     if pagename.startswith("+ ") and "/" in pagename:
00184         generate_summary = True
00185         pagename = pagename[1:].lstrip()
00186 
00187     pagename = wikiutil.normalize_pagename(pagename, request.cfg)
00188 
00189     if choose_html and msg['html']:
00190         content = "{{{#!html\n%s\n}}}" % msg['html'].replace("}}}", "} } }")
00191     else:
00192         # strip signatures ...
00193         content = re_sigstrip.sub("", msg['text'])
00194 
00195     return {'pagename': pagename, 'content': content, 'generate_summary': generate_summary}

Here is the caller graph for this function:

def MoinMoin.mail.mailimport.import_mail_from_file (   request,
  infile 
)
Reads an RFC 822 compliant message from the file `infile` and imports it to
    the wiki. 

Definition at line 201 of file mailimport.py.

00201 
00202 def import_mail_from_file(request, infile):
00203     """ Reads an RFC 822 compliant message from the file `infile` and imports it to
00204         the wiki. """
00205     return import_mail_from_message(request, email.message_from_file(infile))

Here is the call graph for this function:

def MoinMoin.mail.mailimport.import_mail_from_message (   request,
  message 
)
Reads a message generated by the email package and imports it
    to the wiki. 

Definition at line 206 of file mailimport.py.

00206 
00207 def import_mail_from_message(request, message):
00208     """ Reads a message generated by the email package and imports it
00209         to the wiki. """
00210     _ = request.getText
00211     msg = process_message(message)
00212 
00213     wiki_addrs = request.cfg.mail_import_wiki_addrs
00214 
00215     request.user = user.get_by_email_address(request, msg['from_addr'][1])
00216 
00217     if not request.user:
00218         raise ProcessingError("No suitable user found for mail address %r" % (msg['from_addr'][1], ))
00219 
00220     d = get_pagename_content(request, msg)
00221     pagename = d['pagename']
00222     generate_summary = d['generate_summary']
00223 
00224     comment = u"Mail: '%s'" % (msg['subject'], )
00225 
00226     page = PageEditor(request, pagename, do_editor_backup=0)
00227 
00228     if not request.user.may.save(page, "", 0):
00229         raise ProcessingError("Access denied for page %r" % pagename)
00230 
00231     attachments = []
00232 
00233     for att in msg['attachments']:
00234         i = 0
00235         while i < 1000: # do not create a gazillion attachments if something
00236                         # strange happens, give up after 1000.
00237             if i == 0:
00238                 fname = att.filename
00239             else:
00240                 components = att.filename.split(".")
00241                 new_suffix = "-" + str(i)
00242                 # add the counter before the file extension
00243                 if len(components) > 1:
00244                     fname = u"%s%s.%s" % (u".".join(components[:-1]), new_suffix, components[-1])
00245                 else:
00246                     fname = att.filename + new_suffix
00247             try:
00248                 # att.data can be None for forwarded message content - we can
00249                 # just ignore it, the forwarded message's text will be present
00250                 # nevertheless
00251                 if att.data is not None:
00252                     # get the fname again, it might have changed
00253                     fname, fsize = add_attachment(request, pagename, fname, att.data)
00254                     attachments.append(fname)
00255                 break
00256             except AttachmentAlreadyExists:
00257                 i += 1
00258 
00259     # build an attachment link table for the page with the e-mail
00260     attachment_links = [""] + [u'''[[attachment:%s|%s]]''' % ("%s/%s" % (pagename, att), att) for att in attachments]
00261 
00262     # assemble old page content and new mail body together
00263     old_content = Page(request, pagename).get_raw_body()
00264     if old_content:
00265         new_content = u"%s\n-----\n" % old_content
00266     else:
00267         new_content = ''
00268 
00269     #if not (generate_summary and "/" in pagename):
00270     #generate header in any case:
00271     new_content += u"'''Mail: %s (%s, <<DateTime(%s)>>)'''\n\n" % (msg['subject'], email_to_markup(request, msg['from_addr']), msg['date'])
00272 
00273     new_content += d['content']
00274     new_content += "\n" + u"\n * ".join(attachment_links)
00275 
00276     try:
00277         page.saveText(new_content, 0, comment=comment)
00278     except page.AccessDenied:
00279         raise ProcessingError("Access denied for page %r" % pagename)
00280 
00281     if generate_summary and "/" in pagename:
00282         parent_page = u"/".join(pagename.split("/")[:-1])
00283         old_content = Page(request, parent_page).get_raw_body().splitlines()
00284 
00285         found_table = None
00286         table_ends = None
00287         for lineno, line in enumerate(old_content):
00288             if line.startswith("## mail_overview") and old_content[lineno+1].startswith("||"):
00289                 found_table = lineno
00290             elif found_table is not None and line.startswith("||"):
00291                 table_ends = lineno + 1
00292             elif table_ends is not None and not line.startswith("||"):
00293                 break
00294 
00295         # in order to let the gettext system recognise the <<GetText>> calls used below,
00296         # we must repeat them here:
00297         [_("Date"), _("From"), _("To"), _("Content"), _("Attachments")]
00298 
00299         table_header = (u"\n\n## mail_overview (don't delete this line)\n" +
00300                         u"|| '''<<GetText(Date)>> ''' || '''<<GetText(From)>> ''' || '''<<GetText(To)>> ''' || '''<<GetText(Content)>> ''' || '''<<GetText(Attachments)>> ''' ||\n"
00301                        )
00302 
00303         from_col = email_to_markup(request, msg['from_addr'])
00304         to_col = ' '.join([email_to_markup(request, (realname, mailaddr))
00305                            for realname, mailaddr in msg['target_addrs'] if not mailaddr in wiki_addrs])
00306         subj_col = '[[%s|%s]]' % (pagename, msg['subject'])
00307         date_col = msg['date']
00308         attach_col = " ".join(attachment_links)
00309         new_line = u'|| <<DateTime(%s)>> || %s || %s || %s || %s ||' % (date_col, from_col, to_col, subj_col, attach_col)
00310         if found_table is not None:
00311             content = "\n".join(old_content[:table_ends] + [new_line] + old_content[table_ends:])
00312         else:
00313             content = "\n".join(old_content) + table_header + new_line
00314 
00315         page = PageEditor(request, parent_page, do_editor_backup=0)
00316         page.saveText(content, 0, comment=comment)

Here is the call graph for this function:

Here is the caller graph for this function:

def MoinMoin.mail.mailimport.import_mail_from_string (   request,
  string 
)
Reads an RFC 822 compliant message from a string and imports it
    to the wiki. 

Definition at line 196 of file mailimport.py.

00196 
00197 def import_mail_from_string(request, string):
00198     """ Reads an RFC 822 compliant message from a string and imports it
00199         to the wiki. """
00200     return import_mail_from_message(request, email.message_from_string(string))

Here is the call graph for this function:

Definition at line 43 of file mailimport.py.

00043 
00044 def log(text):
00045     if debug:
00046         print >> sys.stderr, text

Here is the caller graph for this function:

Processes the read message and decodes attachments. 

Definition at line 72 of file mailimport.py.

00072 
00073 def process_message(message):
00074     """ Processes the read message and decodes attachments. """
00075     attachments = []
00076     html_data = []
00077     text_data = []
00078 
00079     from_addr = get_addrs(message, 'From')[0]
00080     to_addrs = get_addrs(message, 'To')
00081     cc_addrs = get_addrs(message, 'Cc')
00082     bcc_addrs = get_addrs(message, 'Bcc') # depending on sending MTA, this can be present or not
00083     envelope_to_addrs = get_addrs(message, 'X-Original-To') + get_addrs(message, 'X-Envelope-To') # Postfix / Sendmail does this
00084     target_addrs = to_addrs + cc_addrs + bcc_addrs + envelope_to_addrs
00085 
00086     subject = decode_2044(message['Subject'])
00087     date = time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(mktime_tz(parsedate_tz(message['Date']))))
00088 
00089     log("Processing mail:\n To: %r\n From: %r\n Subject: %r" % (to_addrs[0], from_addr, subject))
00090 
00091     for part in message.walk():
00092         log(" Part " + repr((part.get_charsets(), part.get_content_charset(), part.get_content_type(), part.is_multipart(), )))
00093         ct = part.get_content_type()
00094         cs = part.get_content_charset() or "latin1"
00095         payload = part.get_payload(None, True)
00096 
00097         fn = part.get_filename()
00098         if fn is not None and fn.startswith("=?"): # heuristics ...
00099             fn = decode_2044(fn)
00100 
00101         if fn is None and part["Content-Disposition"] is not None and "attachment" in part["Content-Disposition"]:
00102             # this doesn't catch the case where there is no content-disposition but there is a file to offer to the user
00103             # i hope that this can be only found in mails that are older than 10 years,
00104             # so I won't care about it here
00105             fn = part["Content-Description"] or "NoName"
00106         if fn:
00107             a = attachment(fn, ct, payload)
00108             attachments.append(a)
00109         else:
00110             if ct == 'text/plain':
00111                 text_data.append(payload.decode(cs))
00112                 log(repr(payload.decode(cs)))
00113             elif ct == 'text/html':
00114                 html_data.append(payload.decode(cs))
00115             elif not part.is_multipart():
00116                 log("Unknown mail part " + repr((part.get_charsets(), part.get_content_charset(), part.get_content_type(), part.is_multipart(), )))
00117 
00118     return {'text': u"".join(text_data), 'html': u"".join(html_data),
00119             'attachments': attachments,
00120             'target_addrs': target_addrs,
00121             'to_addrs': to_addrs, 'cc_addrs': cc_addrs, 'bcc_addrs': bcc_addrs, 'envelope_to_addrs': envelope_to_addrs,
00122             'from_addr': from_addr,
00123             'subject': subject, 'date': date}

Here is the call graph for this function:

Here is the caller graph for this function:


Variable Documentation

Definition at line 25 of file mailimport.py.

Definition at line 23 of file mailimport.py.

tuple MoinMoin.mail.mailimport.re_sigstrip = re.compile("\r?\n-- \r?\n.*$", re.S)

Definition at line 27 of file mailimport.py.

Definition at line 324 of file mailimport.py.

Definition at line 319 of file mailimport.py.