This page presents the latest important news related to Weboob.
If you want more, you can check out Planet Weboob.

Weboob 0.j

Posted by Florent Fourcot,

After the summer, it was time to release a new version of Weboob: 0.j is out!

Please carefully upgrade your Weboob installation if you use external tools like munin plugin of personnal scripts. Many class names have been changed, and it breaks the compatibility with tools outside of the official tree. We made this choice to not inherit of the old bad names in the version 1.0.

In other news, the REPL applications are now more user friendly in some circumstances, like when the chosen formatter cannot display all the selected fields. The identifiers and backend names used in CLI applications can now be reduced, like the well know Linux ip command does. In the contrib folder, a plugin is available for the media player XBMC/Kodi. There are four new modules, including Twitter and Wells Fargo, an American Bank.

For developers, Browser2 have been improved with several features. Filters have been improved as well, and it is now possible to use the & and | boolean operators to combine filters. It is more readable than the encapsulation in parenthesis. We made an effort to improve and centralize all the documentation in the development webiste.

There are now 165 modules and 37 applications.

Credits

Changelog

General

  • New module: feedly (CapMessages)
  • New module: oney (CapBank)
  • New module: twitter (CapMessages)
  • New module: wellsfargo (CapBank) (#1430)

Core

  • Rename CapBaseObject to BaseObject (#1424)
  • Rename ICap to Cap (#1424)
  • Ability to use weboob.function as alias to weboob.do('function') (#1425)

Core: repositories

  • Fix HTTP error handling for browser2
  • Use ConfigParser in priority with python2 (#1393)
  • Load browser only when needed

Capabilities

  • Move DateField/TimeField/Delta out of BaseObject
  • Add LBP to currencies
  • Add documentation on object constants

Capabilities: audio

  • Add Playlist and Album classes

Capabilities: audiostream

  • Fix: get_audiostream does not have pattern name (#1626)

Capabilities: dating

  • Add iter_new_contacts method

Capabilities: files

  • Fix repr() and str() on File-based objects

Capabilities: image

  • Remove data field in to_dict method to avoid json crash during conversion

Capabilities: messages

  • Remove required items in Message constructor

Capabilities: travel

  • Do not require an id in constructor

Applications

  • Remove default import of browser1
  • Import debug modules only when needed

Applications: console

  • Remove the import of SSL exceptions
  • Add the default value displayed "upper" in aliases (#1319)
  • Allows shortcuts for modules (#881)
  • Use shortcut of id in interactive mode (#881)

Applications: REPL

  • Allow to browse subfolders with ls
  • Change formatter when it cannot handle all selected fields
  • Introduce the DISPLAYED_FIELDS in formatter
  • Set fields in a consistant way with do()
  • Introduce parse_fields function
  • Use fullid parameter for CapObjects
  • Correct multiple language error
  • Move format_collection from repl to ifromatter
  • Remove the 'inspect' command

Applications: formatters

  • Remove the '*' special fields in formatter
  • Table and Json formatters can write output to a file now (#1412)
  • Handle format_collection with JSON formatter

Applications: boobank

  • Do not crash if the account type isn't in list (#1542)
  • Write the account currency in ofx output

Applications: radioob

  • Manage Albums and Playlists
  • Fix: bug when a radio id contains a dot

Applications: QHaveDate

  • Add tab to send queries

Applications: videoob

  • Improve m3u8 management in download

Browser1

  • Introduce local exception for SSL errors
  • Only load FirefoxCookies as needed
  • Update Firefox versions to latest ESR

Browser2

  • Add more specialized exceptions
  • Allow setting query string params on build_url
  • Matching content with url using is_here
  • Ability to override the flush() method
  • Allow for a custom element finder
  • Add CSV pages
  • Do not crash if total_seconds() is not implemented
  • Fix documentation of nr parameter
  • Update Firefox versions to latest ESR
  • Add support for forms with multiple "submit" elements
  • Allow more flexibility for the submit button parameter

Browser2: ListElement

  • Move ItemListTable-Element outside of page.py

Browser2: filters

  • Overload & and | operators to chain filters (#1426)
  • Split filters in several files
  • Fix filters doctest
  • Force unicode
  • New RawText filter
  • New Base filter
  • New Type filter
  • Date: use default value for empty input
  • Date: properly handle defaults that are not datetimes
  • MultiFilter: allow for a default argument
  • Dict: manage default
  • Dict: ability to use Dict['a']['b']['c'] instead of Dict('a/b/c') (#1426)
  • CleanHTML: manage basestring
  • CleanDecimal: possibility to set custom separators
  • CleanDecimal: set replace_dots default value to False
  • CleanDecimal: do not crash with inputs like NotAvailable
  • CleanText: handle the non-breaking space thanks to the re.UNICODE flag
  • CleanText: add an option to keep (but normalize) newlines
  • CleanText: \t is always in \s so no need to add it
  • CleanText: add tests
  • CleanText: fix re flags usage for Python 2.6 (#1444)
  • Env: add support for a default

Documentation

  • New Home Page
  • Add a "How to contribute" page
  • Add logo/favicon
  • Set more customizations
  • Add instruction for developers missing the first steps (#868)
  • Define backends/modules
  • Add local_run in documentation for developers
  • Fix many docstring issues
  • Change module documentation to learn browser2 (#1451)
  • Add repr on NotAvailable, NotLoaded and _NO_DEFAULT constants to be more readable on doc
  • Import several pages from the wiki
  • Add documentation to report a bug (#873)

Tools: AmericanTransaction

Add a transaction amounts cleaner helper for american banks

Tools: captcha

  • Refactor VirtKeyboard class
  • Add a margin attribut
  • Add a grid based virtual keyboard

Tools: date

  • Add more french dates translations
  • Class methods to convert date[time] objects

Tools: genericArticle

  • Fix unicode warning

Tools: make_man

  • Tell that it was generated automatically

Tools: pyflakes

  • Fix: call of pyflakes on Archlinux (#1404)

Tools: test

  • Fix: call of test.py (#1403)

Tools: yaml

  • Represent weboob date[time] objects as timestamps

Misc: local_run script

  • Allow customizing where the modules are

Misc: setup

  • Support python3 (#1417 #1418 #1419)
  • Add prettytable in dependencies (#929)
  • Configure isort and flake8

Misc: Windows Installer

  • Remove some files
  • Fix bugs in windows installer scripts

Contrib: munin

  • Rename generic-munin to weboob-munin
  • Move all scripts in the same folder
  • Encode and decode ID's in weboob-munin

Contrib: boobot

  • Add command %delquote
  • Fix: %searchquote on unicode strings

Contrib: XBMC/Kodi

  • Add a xbmc/Kodi plugin that interracts with videoob

Modules: arte

  • Fix: Do not crash if 'VDA' fields is missing in json
  • Use M3U8 format instead of HBBTV
  • Fill video.url with NotAvailable if url is not found
  • Improve tests
  • Improve video quality choice
  • Handle arte podcasts
  • Add tests for program categories

Modules: aum

  • Implement iter_new_contacts

Modules: banquepopulaire

  • Strip displayed balance at end of transaction labels
  • Display check number in label (#1027)
  • Fix: remove spaces in IDs (#1368)
  • Support loan payment type

Modules: biplan

  • Handle summer holiday in tests

Modules: bnporc

  • Update order regexp
  • Fix: transfer regexp
  • Remove space in ids

Modules: boursorama

  • Some English fixes in comments
  • Add new certificat hash

Modules: bp

  • Fix: new login image for virtkeyboard

Modules: bred

  • Handle space in account number
  • Switch configuration description strings to unicode

Modules: caissedepargne

  • Force use of TLSv1 on lowsslcheck as the web server support of SSLv3 is broken

Modules: colissimo

  • Fix: New API key for collisimo (#1617)

Modules: cragr

  • Order transactions by date to prevent LinearDateGuesser to be duped by the f*cking website

Modules: creditmutuel

  • Fix: set of debit date for card transactions

Modules: dailymotion

  • Fix: dailymotion mplayer error "No stream found to handle url"
  • Fix: use https for test

Modules: francetelevisions

  • Use filters as classes in chain (refs #1426)

Modules: freemobile

  • Some English fixes in comments
  • Fix date of subscriptions when next month as less days than excepted
    (#1347)

Modules: gdcvault

  • Remove unused import of ControlNotFoundError

Modules: grooveshark

  • Update to match Album and Playlist management in radioob
  • Display users playlists only when split_path length is 0
  • Fix: catch exception when id is not an integer

Modules: hellobank

  • Get default account name if the custom name is empty

Modules: hybride

  • Fix: handle summer holiday in tests

Modules: imdb

  • Some English fixes in comments
  • Use omdbapi instead of imdbapi
  • Fix: site changed

Modules: ina

  • Fix: bad characters in titles (double encoded unicode)

Modules: ing

  • Some English fixes in comments
  • Remove the index on ing for pagination
  • Support coming operations
  • Fix: parsing of 'tomorrow' transaction dates

Modules: izneo

  • Fix: bug in page list

Modules: lcl

Modules: leclercmobile

  • Fix: do not crash if balance is not available

Modules: lefigaro

  • Remove dead code

Modules: meteofrance

  • Fix: site changed (#1390)
  • Fix: call the url that retrieve all the search results (#1431)
  • Raise an exception if forecast param is not a city id (#1433)

Modules: opensubtitles

  • Some English fixes in comments
  • Fix: site changed (#1295)

Modules: pastealacon

  • Convert to Browser2 (#674)
  • Use specialized Browser exception

Modules: pastebin

  • Convert to browser2
  • overload & and | operators to chain filters (refs #1426)
  • Handle limit exceeded warning
  • Fix: crash with Base() and filter chaining

Modules: paypal

  • Get more transactions on paypal (#1405)
  • Retrieve all transactions from the history, merchant and regular account support (#1406)
  • Paypal transactions history fetching with adaptive steps (#1406)
  • Checking if tr contains text
  • Make Paypal module use AmericanTransaction helper.
  • Fix: empty amount. (#1415)
  • Support french dates for last CSV request
  • Ignore canceled transactions

Modules: popolemploi

  • Fix: site is now only availbe using https

Modules: presseurop

  • Presseurop is back! (named now voxeurop)

Modules: radiofrance

  • Fix: FIP radio does not work (#1449)

Modules: sachsen

  • Set the datetime to NotAvailable by default

Modules: senscritique

  • Fix: bug in network selection
  • Fix: set channels and programs parameters in get_event

Modules: societegenerale

  • Fix: certificate changed
  • Fix: certificate updated (#1414)

Modules: sueurdemetal

  • Fix: broken module due to departments containing letters

Modules: tinder

  • Update recs only when needed
  • Fix attribute type

Modules: transilien

  • Adapt to browser2
  • Fix: site changed (#938)

Modules: vimeo

  • Fix: site changed
  • Adapt to browser2
  • Enable search and tests (#1082)
  • Catch HttpNotFound errors

Modules: youjizz

  • Overload & and | operators to chain filters (refs #1426)
  • Use filters as classes in chain (refs #1426)

Modules: youtube

  • Fix: Youtube mplayer error "No stream found to handle url"
  • Fix: is_logged function does not work (#1423)
  • Backport some youtube-dl changes (#1422)

Weboob 0.i.1

Posted by Florent Fourcot,

This is a bug fix release, with the support of python 2.7.7 included. This Python version was released after our major version 0.i, and it introduces a conflict on a private method.

There are no new features.

Changelog

Core: browser

  • Fix bug with python 2.7.7

Core: repositories

  • Fix HTTP error handling for browser2
  • Try to import RawConfigParser from ConfigParser before configparser

Packaging

  • Remove some files of windows installer

Tools

  • Fix call of test.py (#1403)
  • Fix call of pyflakes on Archlinux (#1404)

Weboob 0.i

Posted by Florent Fourcot,

After more than four months of development, Weboob 0.i is out!

The most important news is the successful integration of the Browser2 project. It is a new browser based on python-requests (and not Mechanize), and a lot of tools to help the parsing are included (but not related to python-requests). The number of lines can be divided by two to write a module with Browser2. The documentation should be up to date. Browser2 is tested by the conversion of 24 modules, but is still in development and any feedback is appreciated.

But Browser2 is the tree masking the forest of new features. Instead of developing our keyring for password, we add a new configuration option to call any external tools to read the values. We added 13 new modules, and the console applications have some new cool features. For the future, we worked on the compatibility with Python3 (not ready yet). We are on track for a version 1.0 this year.

There are now 161 modules and 37 applications.

Credits

Changelog

General

  • New Browser: Browser2 is here! See below
  • First steps in Python3 support
  • New GPG key for laurentb (see https://laurent.bachelier.name/gpgtransition.txt)
  • New module: btdigg (CapTorrent)
  • New module: Mailinator (CapMessage)
  • New module: guerrillamail (CapMessage)
  • New module: SensCritique (CapCalendar)
  • New module: imgur (CapPaste)
  • New module: unsee (CapPaste)
  • New module: lut.im (CapPaste)
  • New module: pixtoilelibre (CapPaste)
  • New module: ColisPrive (CapParcel)
  • New module: AlloResto (CapBank)
  • New module: LaCentrale (CapPriceComparison)
  • New module: Groupamaes (CapBank)
  • New module: Tinder (CapDating)

Core

  • Import pkg_resources only when needed, to prevent a hard useless dependency

Core: browser2

  • Add a new browser based on python-requests
  • Full support of SSL
  • Add the option _proxy_ssl for backends
  • New management of Html FORMs
  • New URL class
  • Better debugging (save HTTP requests/responses)
  • Simple support of pagination (@pagination decorator)
  • Introduce ListElement and ItemElement (parser helper)
  • Add a lot of Filters for ItemElement
  • Add TableElement, to help the parsing of a table
  • 24 modules are already using browser2!
  • And a lot more...

Core: browser1

  • Unify no_login and nologin
  • Fix call of logger (can be unset)
  • Move exceptions in weboob.tools.exceptions (compatibility with browser2)

Core: BackendCalls

  • Rewrite BackendCalls with queues

Core: repositories

  • Make GPG ignore local settings

Core: configuration

  • Support for password retrieval from any cli tools
  • Prompt user during configuration of a backend

Core: ManPages

  • Escape more '-'

Core: Parsers

  • Add a xml parser
  • Remove unused/obsolete parsers

Core: DateTools

  • Add ChaoticDateGuesser

Applications: base

  • Add version information in Debug
  • catch ^C on global bcall errors handler
  • Allow to use condition on the fullid
  • Do not display warning message if -n or count is set

Applications: console

  • Fix: masked parameter wasn't considered
  • Fix: password encoding issues
  • Display and not backend.EMAIL

Applications: media_player

  • Add a play_proxy option

Applications: Qt

  • Add keywords in .desktop files (#1356)

Applications: REPL

  • Exclude backends that do not implement the required method (#1325)
  • New obj_to_filename() method (from videoob)
  • Do not fetch the collections when not needed
  • Fix selecting a field in non-interactive mode
  • Use a fast path for ls if -U is set
  • Add documentation of the "ls -d" option
  • Gather collections having the same path
  • Override comp_obj from repl in order to sort by start_date

Applications: formatters

  • Add a json formatter that works on a line level (json_line)

Applications: yamlconfig

  • Allow a default equal to None

Application: boobcoming

  • Add documentation
  • Add icon
  • Use ls/cd commands to navigate into categories
  • Improve collection management

Application: boobsize

  • Add icon
  • Display null values
  • Fix hints in not found error

Application: boobtracker

  • Support colors for issues
  • Create/edit tickets in a text editor
  • Fix email address regexp
  • Fix crash in interactive mode
  • New options --tracker, --priority, --start, --due

Application: parceloob

  • Remove trailing \n
  • Ignore status of not loaded backends
  • Always untrack if the id is in the storage

Application: pastoob

  • Paste binary files with "post_bin FILENAME" and "get_bin ID"
  • Add 'info' command

Application: Qhavedate

  • Ignore error when notes are not implemented on contacts

Application: radioob

  • Fix audio objects search
  • Pick up first available stream when radio provides a playlist (#1345)

Application: translaboob

  • Fix a typo in the documentation

Application: videoob

  • Braces-enclosed tags are replaced with data
  • Play/info: ability to give several videos
  • Add command 'videoob playlist download'
  • Add command 'videoob playlist play'
  • Move obj_to_filename() into ReplApplication
  • Use wget to download m3u8 videos

Application: webcontentedit

  • add a -r parameter to get command to precise revision id

Application: wetboobs

  • Remove ICapGauge

Capabilities: Base

  • Change currencies integer constants to ISO code strings
  • Do not require an id, because of ItemElement which constructs objects
    h4. without arguments
  • Introduce find_object

Capabilities: CapWeather

  • Fix: zero temperature can not be displayed (#1333)
  • Accept date objects for the forecast

Capabilities: CapCalendar

  • Raise NotImplementedError on "attends" method
  • Add TELE category

Capabilities: CapBugTracker

  • Fix some conversion warnings

Capabilities: CapBank

  • Add Account.TYPE_CARD
  • Add TransactionsElement and TransactionElement (helpers with browser2)

Capabilities: CapContact

  • Factorize aum's Contact.get_text() and boobmsg formatter

Capabilities: CapGeoloc

  • Remove unused ipaddr field

Capabilities: CapMessage

  • Avoid warnings in GenericNewspaperBackend and GenericNewsPage

Capabilities: CapPast

  • Show a user-friendly error message for CapNotFound

Capabilities: CapTracker

  • Add trackers and priorities fields to Project
  • Add start, due, tracker and priority fields to Issue

Modules: banquepopulaire, bnporc, bp, cic, cragr, creditdunord, creditmutuel, ing, lcl, societegenerale

  • Improvements on transaction and account types detection

Module: AmericanExpress

  • Fix wrong date guessing
  • Increase timedelta to 90
  • Check if a card is valid
  • Set card balance to Account.balance instead of Account.coming

Module: Apec

  • Updates reflecting site changes

Module: Apivie

  • Fix getting an account from list
  • Fix parsing of negative transactions

Module: Arte

  • Encode UTF-8 strings in search
  • Fix naming of collections
  • ArteLive: new website
  • Improve the parsing of dates
  • Fix bug when VDA field not found in json file
  • Fix problem on extension

Module: AxaBanque

  • Adding support for sub accounts
  • Fix parsing of accounts without amount

Module: Banque Accord

  • Add icon
  • Fix parsing of negative amount
  • Fix detection of card pages
  • Fix virtual keyboard
  • Add a hash for symbol 5 on virtkeymap
  • Adding support for Leroy Merlin
  • Support LOAN accounts
  • Upgrade to browser2

Module: Banque Populaire

  • Fix: parsing of accounts (site changed)
  • Fix: parsing of accounts on specific regions
  • If an account type is not supported, raise NotImplementedError

Module: Biplan

  • Fix bug while parsing start/end time
  • Updates reflecting site changes
  • Upgrade to browser2

Module: Bnporc

  • New certificate
  • Fix encoding issues
  • Disable unused parameter rotating_password
  • Change the domain to entreprises.bnpparibas.net for BNPEnterprise

Module: Boursorama

  • Fix crash when there is no ID on transaction
  • Handle card history

Module: BP

  • New certificate
  • List market account (#1298)
  • An account type must not be None

Module: Bred

  • Support dispobank
  • New certificate

Module: CanalPlus

  • Fix browsing categories (site changed)

Module: Carrefour Banque

  • Upgrade to browser2
  • Fix: Site changed

Module: CCI

  • Upgrade to browser2

Module: Citelis

  • New icon
  • Fix: detection of login errors

Module: cmb

  • Change a function call to avoid useless None
  • Upgrade to browser2

Module: cmso

  • Setting encoding to iso

Module: Colissimo

  • Add icon
  • Fix license

Module: Cragr

  • Fix parsing dates in labels (compatibility with Perigord)

Module: Credit Du Nord

  • If type is not found, set TYPE_UNKNOWN instead of None
  • Add "Banque Rhône-Alpes"

Module: Credit Mutuel

  • New icon
  • Upgrade to browser2
  • Fix parsing of user private accounts
  • Fix crash on unknown accounts types

Module: cuisineaz

  • Fix cuisineaz preparation and cooking time (site changed)

Module: Dailymotion

  • Fix empty fields in dailymotion plugin
  • Change extension from flv to mp4
  • New icon
  • Add support for videos available at kids.dailymotion.com
  • Fix the search (site changed)

Module: Delubac

  • Support LCR
  • Fix navigation

Module: DLFP

  • Fix logout (site changed)

Module: DresdenWetter

  • Upgrade to browser2

Module: Fortuneo

  • New certificate
  • Do not override rdate with operation date

Module: France Televisions

  • Upgrade to browser2

Module: Freemobile

  • Fix date of subscriptions when next month has less days than expected (#1347)
  • Upgrade to browser2

Module: googletranslate

  • Fix : site is now only available using https

Module: grooveshark

  • Fix: no more field AvgRate (site changed)

Module: HSBC

  • Fix: support of the SecureKey
  • Upgrade to browser2

Module: hybride

  • Upgrade to browser2

Module: INA

  • Site changed

Module: indeed

  • Upgrade to browser2

Module: ING

  • Update regexps used to detect categories (site changed)
  • Upgrade to browser2
  • Support pagination in iter_bills
  • Do not crash on invest objects when quantities are >999

Module: Kickass

  • Fix title getter (site changed)

Module: LeclercMobile

  • Do not crash if the subscription is closed

Module: LeFigaro

  • Fix: site changed
  • Fix the tests

Module: LCL

  • New maintener
  • Support changes in login screens
  • Get more history
  • Remove logging checking on the contract page

Module: Liberation

  • Add icon

Module: lolix

  • Fix the size of the icon

Module: mediawiki

  • Fix encoding error

Module: MeteoFrance

  • Strip the text before to return it

Module: OKC

  • Add the visit to a profile
  • Add a profile walker
  • First sent message is configurable
  • Do not crash on removed contacts
  • Do not crash on 'like' messages

Module: pastealacon

  • Upgrade to browser2

Module: Paypal

  • Certificate changed (three times)

Module: piratebay

  • New domain (thepiratebay.se)

Module: Pole Emploi

  • Updates reflecting site changes
  • Fix parsing of dates
  • Fix bug in url encoding
  • Fix place selection

Module: Poivy

  • Upgrade to browser2
  • Support pagination for the history

Module: Redmine

  • Add Issue.fields attribute to support custom fields
  • Support new versions of redmine
  • Use the right method to get project
  • If a category does not exist, try to create it
  • Fix compatibility with redmine 2.4
  • Support start/end/tracker/priority
  • Fix finding the control to add a note

Module: RegionsJob

  • Add icon
  • Upgrade to browser2

Module: sachsen

  • Upgrade to browser2

Module: Société Générale

  • Display card as accounts (#1362)

Module: Sueurdemetal

  • Add date_end parameter in get_concerts_date call
  • Fix date and category filters

Module: voyagessncf

  • If station ID is not found, try to look for it as a name
  • Forge a random IP source address to avoid redirection to other website (#1327)

Module: vlille

  • Upgrade to browser2

Module: Weather

  • Fix: there are not always ten days of forecast (#1343)

Module: youjizz

  • Handle videos longer than 59 minutes
  • Support pagination
  • Upgrade to browser2

Module: youtube

  • Fix: site changed
  • Fix: unexpected argument to BrokenPageError

Tools: boilerplate

  • Add 'cap' recipe to prepare methods
  • Upgrade to browser2

Tools: local_install

  • Run tools/stale_pyc.py before local installation

Tools: pyflakes

  • Ignore 'except' in a comment

Contrib: boobot

  • Add quoting commands
  • Fix unicode issues

Contrib: Windows Installer

  • Better script to generate the .exe
  • Better management for 64 bits systems