This page presents the latest important news related to Weboob.
If you want more, you can check out Planet Weboob.

Weboob 1.2 is out

Posted by Florent Fourcot,

Happy new year! A new release of Weboob is available for download and installation. The changelog is huge. Even if many patches are fixes for modules, we have very interesting new features in core and browser.

For example, modules, browsers and pages can now inherit from code of others modules. That will reduce number of boilerplate for some modules, and allow a better modules management. We support more types of pages (sometimes of data format that you will not see every days), and it's easier to get HTML sources of modules. You should also have a look on the JSON encoder if you need to read Weboob output from some applications.

From a user point of view, Qt5 migration should be a great improvement. We have also a large numbers of new modules and new features for older modules. Last but not least, the split between CapBill and CapDocument should allow to build better applications to manages documents on various websites.

It's not code related, but we moved our git repositories to gitlab and we plan to migrate step by step all features (like issues tracker) hosted today on redmine to it. The bad news is that we broke URL compatibility, you should update all git URLs related to Weboob in your locals repositories. One of the good news is that we now use gitlab-ci integration, and we should be able to provide very soon a better overview of real module states, based on our tests.

As promised, the support of python 2.6 has been dropped in this release. Large part of code and modules will not work on python 2.6. We hope to move to python 3.5 before the end of 2017.

Credits

General

  • Add inheritance for modules, browsers and pages
  • Add gitlab-ci integration
  • Move all Qt applications to Qt5
  • New application QBooblyrics
  • New urls for source repositories: https://git.weboob.org/

General: removed modules

  • Remove btdigg module, website is dead

General: new modules

General: core

  • Add BackendsCall.stop() method to stop jobs
  • Add a 'requests' system to replace the primitive callbacks system
  • Check version before to run update request in repositories
  • Fix core unittests and some pyflakes warnings
  • Make as much modules unittests as possible run without backend
  • Add WebNip.load_or_install_module which does not install module (for compatibility with abstract objects)
  • Some fixes for windows compatibility
  • Force keyring generation with gpg1

Documentation

  • Fix some typo in browser documentation
  • Clarify absurl behavior
  • Don't add applications/index in index.rst (empty page)
  • Don't use :any: special keywork for old system compatibility
  • Update favicon path
  • Add contribution guide
  • Rewrite Weboob doc guides according to latest changes in CI and modules

Browser

  • Detect downgrade from HTTPS to HTTP and raise exception
  • TableElement now handles regexp
  • Do not store page url if there is no current location
  • Use urllib3 embedded in requests first
  • Update firefox ESR profile to the last version
  • Keep order of POST data
  • Add XLSPage, PDFPage and GWTPage classes
  • Catch TooManyRedirects after reloading cookies from storage
  • Do not use (and remove) global logger setting 'save_responses' to know if we save responses or not
  • Fix some invalid encoding names (#2602, #3522)
  • Introde CacheMixin class to store responses
  • Store the request object in Form class (allow to modify it on the fly)
  • Add xpath funtions star-with and ends-with
  • Introduce PartialHTMLPage for pages wit multi-roots

Browser: URL

  • Fix a crash when a pattern use parameters with the same name than in another pattern
  • Add unit test for URL with multiple patterns and same parameters subset
  • Add an option to send headers in go() method

Filters

  • RawText: can now optionally retrieves text from sub-elements
  • Date/Datetime: add fuzzy option
  • Async: read result.page from loaded_page
  • TableCell: can now iterate on <th> elements
  • Add Upper and Capitalize filters
  • Add Currency filter

Elements

  • DictElement: Handle sub-dictionaries
  • ItemElement: Ignore items if the condition fails
  • ListElement: check condition before iteration

Old browser

  • Allow monkey patch to disable SSL certificate check
  • Better SSL support (import more recent TLS versions if available)
  • Fix https downgrade detection

Exceptions

  • Add ActionNeeded exception
  • Add NoAccountsException
  • Move ModuleInstallError to Exceptions module

Tools

  • Introduce LimitedLRUDict to not cache too much values
  • Add a WeboobEncoder class
  • Add get_pdf_rows helper

Tools: Backend

  • Save responses if _debug_dir backend configuration is set
  • Add optional 'klass' parameter for Module.create_backend

Tools: bank

  • Add tools about IBANs and French ribs

Tools: date

  • Add Italian translation
  • Fix date translation for values with \b

Tools: formatters

  • Fix TableFormatter with lists of different lengths
  • Protect separator contained into values for CSV output (#2614)
  • Simplify CSV formatter with a call to csv python module
  • Fix use of -O parameter for csv formatter
  • Set ";" as csv default separator (#3378)

Tools: tests

  • Add skip_without_config decorator
  • Add is_backend_configured method to BackendTest class

Tools: ValueBackendPassword

  • When callback returns None, convert to empty string

Tools: VirtKeyboard

  • Add parameterizable output code separator

Capabilities

  • Add an url attribute to BaseObject
  • Add many currencies (turkish lira, romania new leu, colombian peso, etc)
  • Fix few repr/__str__ returning unicode
  • Add BoolField class

Capabilities: Bank

  • Add description for 'coming' field on Account objects
  • Add PEE, PEA, PERP, PERCO, RSP, ARTICLE_83 and more account types
  • Add CARD_SUMMARY and DEFERRED_CARD transaction types
  • Add portfolio_share field to Investment
  • New fields for Investment to handle currencies
  • Add diff_percent field on Investment
  • Don't crash on empty raw values
  • Split transfers related methods to a new capability: CapBankTransfer
  • Update recipient object
  • Deep rework of transfer API, change modules and applications

Capabilities: Contact

  • Add Advisor class, used in many bank modules

Capabilities: Document / Bill

  • Major (and API breaking) change (all modules are updated)
  • Update all Bill modules to use url field instead of private field
  • Rename field "deadline" to "duedate"
  • Add field "income" to Bill

Capabilities: File

  • Add license field

Capabilities: Housing

  • Add url in HousePhoto repr method
  • Add price_per_meter field

Capabilities: Message

  • Create a genericnewspaper module with no capability that will be extended by other modules

Capabilities: Image

  • Move Thunbmail class to Image capability
  • Fix many modules to use Thumbnail instead of BaseImage

Capabilities: Lyrics

  • Remove useless constructor for SongLyrics objects

Capabilities: PriceComparison

  • Add Price not found error

Capabilities: Recipe

  • Do not crash if picture_url is an empty string
  • Handle decimal quantities in krecipe exports

Applications: weboob script

  • Add "update" command

Applications: console

  • Use self.do instead od self.weboob.do in get_object_list
  • Strip left and right condition (we now support rules lines like --condition 'amount > 2000')
  • Disable addresses (to much SPAM)

Applications: all Qt applications

  • Use new-style signals and slot
  • Add Qt5 package in our tools
  • Port all applications to Qt5
  • Add "wait" parameter to stop method

Applications: boobank

  • Display sub-totals for each currency (when several currencies are detected)
  • Fix error display in budgea command
  • Increase timeout of budgea command
  • Do not take of loan accounts in total
  • Display loan accounts label in blue
  • Use encoding aware custom print()
  • New advisor command with custom formatter

Applications: comparoob

  • Add defaultcount management
  • Improve products management

Applications: flatboob

  • Improve display if Area is not found

Applications: monboob

  • Add full thread history in headers (fix some bad detection one MUA)

Applications: videoob

  • Correctly handle non complete url in m3u8

Applications: qcineoob

  • Sort torrent search result list by seeders
  • Display backend icon for each search results
  • Set tab text max size and add tooltip
  • Add keyboard shortcut (alt+left) to go back
  • Tab text is now updated on actions

Applications: qcookboob

  • Display backend icon for search results
  • Add keyboard shortcut (alt+left) to go back
  • Better tab text/tooltip management
  • Fix bug when opening in a new tab, bad is was given

Applications: qflatboob

  • Load all fields in widget list (help to rank adverts)
  • Display price per meter ratio

Tools: boilerplate

  • Improve arg signature of methods generated by CapRecipe
  • Make u8 idempotent
  • Don't add failing test by default

Tools

  • local_run: fix script when PYTHONPATH is not set
  • run_tests.sh: fix wrong exist status when specifying a single backend

Packaging

  • Add a command to get the requirements with setup.py
  • Add cssselect to the list of requirements
  • Ensure that requests version is >= 2.0

Contrib

  • Boobot: add tasks management
  • Boobot: fix compatibility with iso-8859-15 messages
  • Boobank_indicator: do not include loan account in total
  • Add imm-o-matic script

Modules: 750g

  • Update module to use new website
  • Do not crash when there is only one or no result

Modules: adecco

  • Update module to use new website
  • Skip test if backend is not configured
  • Add default configuration for tests

Modules: agendaculturel

  • Website migrated from http to https
  • Skip test if backend is not configured
  • Add default configuration for tests

Modules: agendadulibre

  • Skip test if backend is not configured
  • Add default configuration for tests

Modules: allocine

  • Update favicon

Modules: allrecipes

  • Uses API instead to parse website

Modules: amazon

  • Handle prices that occur twice on the page (#2057)
  • Fixed login after website update (#2057)
  • Another discount type (#2057)
  • Added 'Free Shipping Promo' bill detailisation keyword. (#2057)
  • Force label to document
  • Fixing amazon : sometimes tva or other amounts arent available
  • Use new login link id

Modules: amazonstorecard

  • Improved stability (retry on error) (#2115)
  • Major website update (#2115)
  • Parse recent transactions (#2115)
  • Handle amounts more than ,000 (#2617)
  • Parse payment due date; allow transactions with empty id (#2617)

Modules: americanexpress

  • Remove unused constants
  • Rewrite to browser2 and remove old code
  • Fix date parsing
  • Update dateguesser range to one year
  • Handle account suspended page
  • List multiple accounts
  • Don't report canceled accounts without history
  • Fix div id with single card

Modules: ameli

  • Update module after site changes

Modules: amelipro

  • Fix tests calls

Modules: apec

  • Fix the way experience and contact fields are filled
  • JSON api format has been updated
  • JSON call got updated
  • Handle accentuated characters in search
  • Add default configuration for tests

Modules: apivie

  • No more fake ids on apivie transactions

Modules: arte

  • Do not crash while trying to handle a bad url
  • Handle an other type of web page in cinema.arte
  • Fix Debian bug #831609, correctly handle m3u8 url
  • Many little fixes
  • Handle arte creative
  • There are some duplicate videos in creative pages
  • Fix arte cinema (module was not able to find video url anymore)
  • Add default configuration for tests

Modules: aum

  • Remove duplicate entry key in AUM module

Modules: axabanque

  • Fix no history on card
  • Handle ACCOUNT_TYPES
  • Fix accounts typing
  • Port to browser2 and adding investments
  • Don't crash in case of customer error
  • Check when user need to change password
  • Fix problems when checking error
  • Fix asv not being fetch and portfolio
  • Fix bug going on agipi page
  • Get account IBANs and fix typing
  • Fix investments on market account
  • Fix accounts from tabs and more investments
  • Fix url change on invest
  • Fix handling pel type
  • Support new login and some changes
  • Fix: password can be only digits
  • Support the new website for investments
  • The login can now be an email
  • Accounts numbers may be written differently
  • New URL for accounts list
  • Some pages return overlapping accounts, skip duplicates

Modules: banqueaccord

  • Support new login page

Modules: banquepopulaire

  • Handling currency for cards
  • Adding currency for card
  • Handle crash on natixis website
  • Removing balance information in raw
  • Fix missing label in history
  • Set Account.iban to NotAvailable if user hasn't subscribed to online documents
  • Fixing banquepopulaire : No IBAN for checking account
  • Fix wrong account type
  • Fixing account type for "Fructi Selection Vie"
  • Adding more life insurance support
  • No IBAN on investment and life insurance
  • Add support for Banque de Savoie.
  • Check number in raw instead of label
  • Handle different label
  • Investments on savings account
  • Port to browser2 the easy way (without filters and so on)
  • Fix problem with unicode cookies
  • Test unavailable service on all pages
  • Raise NotImplementedError for new natixis pages
  • Use absurl for regular pages after having visited market
  • Update navigation token even on error pages
  • Fix crash on some pages without token

Modules: barclays

  • Support investments on life insurance accounts
  • Updated cards handling
  • Fix account type detection to fix card linking
  • Move to browser2 and support IBAN on pdf

Modules: bforbank

  • Adding pagination for bforbank
  • Don't crash on no IBAN for checking account
  • Fix reading when IBAN is not available
  • Add a favicon
  • Use spirica module for life insurance
  • Implement investments/history for MARKET accounts with lcl

Modules: blablacar

  • Fix departure date regexp
  • Fix parsing date
  • Fix: site got updated
  • Add a favicon

Modules: bnporc

  • Don't crash without simplejson (support other json modules)
  • Add support PEA liquidity account type
  • Support one more page to refresh password
  • Get a year of history
  • Correctly decode IBANs
  • Rebuild IBAN only if it isn't already one
  • Catch wrongpass
  • Print temporary password to allow rescue connection on website
  • Fix: site changed
  • Adapt to support CapBankTransfer
  • Try harder to connect on the website
  • Add url to renew password
  • Fix BNPEnterprise: add leading zero when IBAN key is < 10
  • New website for BNPEnterprise, rebuild all module
  • Fix bnp enterprise: libelle not loaded for some reason, so take the type libelle on this case
  • Fix BNPEnterprise: swith date and rdate, move Account import
  • Fix CleanDecimal not always working
  • Catch change password page and raise ActionNeeded
  • Prevent date overlap in history search
  • Transfer fix on bnp
  • Categories on recipients
  • Handle transfer unavailable on fetching IBAN and recipients
  • Raise BrowserUnavailable on 1001 error code
  • Add transfer.account_balance
  • Add advisor support
  • Add a cache for account_list
  • Don't overwrite browser.page with something which can be a Response
  • Use response.json() to parse response, to prevent encoding issues

Modules: boursorama

  • Better fall back on old website
  • Skip first visit page before falling back on old webiste
  • Better newwebsite url
  • Fix unitprice parsing
  • Support of Bourso new website and use Browser2
  • Add support of loan accounts
  • Fixing class for transaction label changed
  • Handling coming card
  • Add a condition to prevent crashs
  • Fix label parsing
  • Update regexp for TYPE_ORDER
  • Fix cards transactions link
  • Handle wrong pin code
  • Fix loan, market, valuation etc ..
  • Two-step-auth: fix request if browser has been shutdown
  • Detect when PIN tries limit has been reached
  • Fix checking of errors
  • Handling when no informations is avalaible in investment
  • Fix in case there are divs into accounts list
  • Handle new bourso card ID
  • handle blocked page1
  • Raise wrongpass on blocking secret infos update page
  • Handling new info profil page
  • Take custom-id if regular ids are not present
  • Fix same transactions on different accounts
  • Fix cards rdate parsing
  • Handle ord accounts
  • Fix pea history parsing
  • Fix some nav issues
  • More accurate url and valuation calculation
  • Adding incident page
  • Fixing wrong inheritance
  • Handle no link on market history
  • Fix history parsing
  • Parse transactions label back again
  • Try to be easier with bourso website
  • Do not iter coming on li and market accounts
  • Add a cache for accounts
  • Handle new history page type
  • Increase HTTP timeout, the website can be slow
  • Better typing to avoid crash
  • Do not crash on ClientError
  • Try to update balances of accounts before parsing
  • Try to fix XMLSyntaxError
  • Don't rash when no id are available (on some foreign accounts)
  • Catch XMLSyntaxError
  • Fix virtkeyboard
  • Don't call need_login twice
  • Fix page not being handled by the right class
  • Date might be missing
  • Fix unauthorized on virtkeyboard with old stored cookies
  • Fix cookie delation after going on login page
  • Fix card transactions request
  • Fix skiping not boursorama's accounts
  • Fix card accounts ids
  • Handle more cases where client need to log in on website
  • Handle more login page
  • relog on random disconnection
  • Fix coming parsing
  • Add IBAN page
  • Fix market accounts transactions date parsing
  • Fix useless nav and wrong webid leading to wrong IBAN
  • Fix transaction amount parsing
  • Fix crash on parsing amounts
  • Handle url with correct page
  • Type more loan accounts to prevent crash on history
  • Exclude deffered_card transactions of checking accounts
  • Add new param to login
  • Fix crash on expert derive account

Modules: bouygues

  • Fixing bouygues module which works with b&you
  • Fix date parsing
  • Fixing bouygues module multiple accounts
  • Don't crash for accounts without document
  • For some reason, bills might be duplicated on website
  • Fix: https is now mandatory for documents URL
  • Fix: sometimes download document crash when reference is not specified

Modules: bp

  • Handling wrong website selection
  • Add wrongpass page
  • Fix duplicate accounts with better id parsing
  • Fix regexp to get accounts ids
  • Support CapBankTransfer new API
  • Accounts are now in EUR by default
  • Handle deferred card
  • Accurate messages if users are on wrong website (between pro and perso websites)
  • Don't crash when no accounts are available
  • Port to browser2 (the easy way)
  • List loan accounts
  • More support of transfer (double auth not handled)
  • Fix pro browser working with StatesMixin
  • Reimbursed loans have an empty amount string
  • Parse life insurance investments and history
  • Fetch history for retirement contracts
  • Skip loan account with "unavailable data"
  • Fix BrowserIncorrectPassword with par/pro mix
  • Fix statesmixin on expired session
  • Handle internal recip not in accounts list
  • Add a cache for accounts_list
  • Fix a crash on unavailable pages
  • Skip more useless LOAN accounts
  • Add more regexp for transaction types
  • Add advisor support
  • Catch no accouts

Modules: bred

  • Cut password to the first 8th char
  • Fetch IBAN only for checking accounts
  • Transactions typing
  • Do not yield not real market accounts
  • Fix duplicate id accounts

Modules: caissedepargne

  • Fix pagination on history
  • Fix investments on multiple market accounts
  • Fix come back on synthesis
  • Handle loans caisse depargne
  • Fixing when not loan account is present
  • Better check if we are on wrong page
  • Detect and bypass useless message page
  • Fixing website failed redirecting
  • Handle delayed debit card and cleaning module
  • Browser handles technical error page (instead of code in iter_* functions)
  • Add certhash
  • Fix account valuation_diff parsing
  • Do not yield anything on market cgu
  • Guess IBAN only if this is a correct RIB
  • Fix login
  • Fix encoding error
  • Fix when there are multiple choices of spaces (between pro and personal)
  • Remove LoginPage and do only direct requests
  • Better wrong pass handling
  • Fixing can't leave credit page on old website
  • Fix navigation on history
  • Catch wrongpass
  • Life insurance might not be available for consultation
  • Adding lifeinsurance's history
  • Fix website being awful
  • Fix infinite loop on no accounts
  • Only keep relevant wrongpass (and return BrowserUnavailable for other errors)
  • Fallback on pro domain when loggin on personal fails if pro exists
  • Invest date might be unavailable
  • Better handle of real no accounts
  • Read multiple cb tabs
  • Bump to browser2
  • Fix: sometime link is a list, so fix this
  • Fix: an 'href' who takes himself for an 'a' tag
  • Fix: on cenet website, sometime a year is not available, so try every months
  • Fix: on garbage page sometime link is not available, due to an ActionNeeded
  • Fix: if garbage go_back_link is not available, skip this step
  • Remove deprecated exception, use new ones
  • Add support of advisor

Modules: carrefourbanque

  • Manage more accounts, investments
  • Fix portfolio on some modules

Modules: cic

  • Improve account typing
  • Update cic sources from credit mutuel
  • Support CapBankTransfer new API
  • Cic now inherits from creditmutuel
  • Adding advisor for cm/cic

Modules: citibank

  • Fixed login after website update. (#2098)
  • Switching to javascript library
  • Switching to new base url
  • When parsing a date in a statement, try both leap and non-leap years (#2098)
  • Handle negative and positive account balance (#2098)

Modules: cmb

  • Handling agricultor website
  • No more crash on no account
  • Bump to browser2 and more (more accounts, investments, coming...)
  • Use AbstractBrowser instead of symlinks to cmso

Modules: cmso

  • Fix account type
  • Fix wrongpass detection
  • Don't raise BrowserUnavailable when no savings accounts
  • Don't crash when no accounts are available in pro website
  • Manage multiple areas
  • Don't crash without accounts
  • Index were not unique
  • Don't crash when no IBAN for checking account
  • Support cmmc website
  • cmb/cmso/cmmc: support loans
  • Return on subscription before switching area to prevent 500
  • Handle 500 when no history on market account
  • Have useful account ids
  • More history is now recovered
  • Add a forgotten return in iter_history
  • Getting a year of history
  • Use open instead of random async
  • Better id on market accounts
  • Fix crash when id is not found on market accounts
  • Handle multiarea on cmso pro
  • Handle no account in area
  • Fix regexp for some account ids
  • Start fetching investments
  • Fill 'areas' attribute outside of do_login (to work with blackbox)
  • Fix investments with non investment accounts
  • cmso pro: prevent infinite loop

Modules: colisprive

  • Add a favicon

Modules: colissimo

  • Get full history

Modules: cragr

  • Fix broken perimeters
  • Get rid of not required assert
  • Account type on label before section title
  • Update label and balance on market websit
  • Fix bad home url and no account link for market accounts
  • Fix when cards and market accounts
  • Do not globally crash if we can't go to the market page
  • Fix current_perimeter and no link for market
  • Fix guessing of market links (use real hostname, not proxy's one)
  • Fix investment parsing and add vdate
  • Type and history of ESPE INTEG
  • Fix cards history navigation
  • Fix PEL history
  • Fixing cragr investment
  • Fix regression with cards navigation
  • Handle pagination on cragr CardsPage
  • Fix invest parsing
  • Do not crash on cards on some weird cases
  • Raise wrongpass on cgu
  • Fix investment parsing
  • Correctly handle the first connection page
  • Fix bad detection of current perimeter
  • Truncate passwords to first six chars
  • Remove trailling chars in isin
  • Fix encoding in investment code
  • Don't try to detect perimeter on not handled page
  • Handle perimeters having the same name
  • Handling currency on multiple card
  • Truncate username to the first 11 chars
  • Add new DAV TIGERE accounts
  • Fix infinite loop on wrong pass
  • Fix cards
  • Fix transactions parsing (new column)
  • Add account CPTEXCPRO
  • Fix multiple cards accounts
  • Update ordering date in get_history, it's useless and the link might be dead.
  • Fix transactions parsing on some accounts type
  • Fix bad isin
  • Handle life insurance on bgpi website
  • Fix some lifeinsurance
  • Adding pagination on accounts page
  • Fix parsing of unitprice on some market accounts
  • Fix portfolio_share on bgpi invest
  • Fix crash when no investments in BGPI
  • Fix listing card with pagination
  • Fix get of IBAN
  • Adapt to new CapBankTransfer API
  • Don't try to read history on market accounts
  • Fix detail unavailable on market accounts
  • Fix parsing of valuation_date of market invest
  • Seems like cragr normandie is not using new login method anymore
  • Update webid of deferred card to avoid duplicates
  • Fix sublabel parsing, sometimes there's no sublabel parent

Modules: creditcooperatif

  • Add coming card page
  • Limit password to 12 chars
  • Fix xpath issue
  • Fix crash on technical error on website
  • Add IbanPage support
  • Fix crash on notavailable iban page

Modules: creditdunord

  • Add bank SMC support
  • Add bank nuger support
  • Fix transactions parsing on rhonealpe
  • Unify regular and pro accounts cards handling
  • Fix deferred_card typing
  • Dectection of wrongpass
  • Fix duplicate accounts
  • Skip accounts without details
  • Fix bad isin like FR0010411884
  • Handle no account
  • Move to browser2
  • Handle loans on pro website
  • In iter_* methods, raise AccountNotFound if account is not found, instead of crash
  • Fix navigation on website to list loan accounts
  • Fix parsing of transactions without value date
  • Fix parsing of euro funds
  • Add loan typing
  • Fix navigation on website to get transactions
  • Fix creditdunord transaction date parsing
  • Some dates are not available instead of a parse error

Modules: creditmutuel

  • Fix history and investments of life insurance accounts
  • Fix life insurance
  • Fix coming for cards
  • Fix when unitprice not available
  • Set default balance value to NotAvailable
  • Handle fleet cards
  • handle page to detect new website
  • Fix date parsing for cards
  • Raise skipitem instead of parse error
  • Fix amount commission of non euro fleets transactions
  • Handle hidden parts of labels
  • Fix isin parsing
  • Adding IBAN for new website
  • Parse life insurance's transactions dates as French dates;
  • Get IBAN based id
  • Handle pagination
  • Fix crash on more transactions
  • Only use last card differed for coming
  • Support cb diff
  • Adding cards to creditmutuel's new website
  • Fix setting deleted to transactions
  • Fix parsing of differred_date
  • Don't crash on empty city field
  • Use new CapBankTransfer API
  • Fix id and IBAN of accounts on old website but displayed on new accounts page
  • Fix date parsing for coming
  • Handle real no accounts on cic
  • Fix currency on lifeinsurance accounts
  • Fix crash when there is no bank accounts
  • Fix card names and set coming balances
  • Fix multiple month of history on cb differed
  • Correctly handle lifeinsurance accounts on cic
  • Handle newhome page to detect new website
  • Fix original amount of card transactions on new website
  • Get now about 6 months of history on new website
  • Handle loan history page
  • Handle page and fix crash on history on some accounts
  • Handle another user page
  • Fix card history link
  • Fix transfer on the new website
  • Add support of creditmut pro and add transfer category
  • Fix regexp for URL of subbank
  • Fix finding account and recipient in init_transfer
  • Skip wrong IBAN instead of crash
  • Limit transfer label to 27 char instead of raising error
  • Add cards on new website with tiers page
  • Add transfer.account_balance
  • Don't crash if a transfer has no label
  • Truncate label checking
  • Strip transfer.label to compare with bank return
  • Detect ActionNeeded after login
  • Prefer None accounts_list over missing attribute
  • Skip debit card
  • Add support of life insurance on new website
  • Add advisor
  • Parse balance and currency of cards in some cases
  • Raise wrongpass with error messages

Modules: cuisineaz

  • Update to the new website site changed

Modules: dailymotion

  • Fix module and bump to browser2
  • Get video from page url

Modules: delubac

  • Detect bad password
  • Add basic account types
  • Get IBAN
  • Fix transactions parsing

Modules: dhl

  • Use dhl.com's JSON API
  • Add support for both DHL Express and Deutsche Post DHL
  • show URL in event description

Modules: dlfp

  • Populate url attribute messages and thread objects
  • Always fill the parent attribute
  • Disable wiki tests when the backend is not configured

Modules: dresdenwetter

  • Skip test if backend is not configured

Modules: edf

  • Bump module to Browser 2 and fix parsing
  • Add edfpro website
  • Add a favicon
  • Always set vat attribute to to NotAvailable
  • Remove bad tests in edfpro folder
  • Ignore duplicates

Modules: entreparticuliers

  • Fix parsing : site changed
  • Correctly handle source
  • Fix url field filling
  • Support new price_per_meter attribute of CapHousing
  • Fix bug when response is empty
  • Add a 100km default rayon value
  • Do not crash if area is unknown
  • Add a favicon

Modules: explorimmo

  • Support new price_per_meter attribute of CapHousing
  • Fix bug in town names' encoding

Modules: feedly

  • Update google login state
  • Remove need_login decorator for get_unread_feed

Modules: fortuneo

  • Update certhash (#2550)
  • Detect wrong passwords
  • Fix duplicate of accounts
  • Don't try to get IBAN for checking account (token sent by SMS to customers)
  • Replace BrowserIncorrectPassword with ActionNeeded on renew password
  • Port to browser2
  • Limit max time site wants us to wait in async requests
  • Fix separators cleaning for decimals

Modules: francetelevisions

  • Ignore duplicate in videos search result

Modules: freemobile

  • Fix subscription listing (site changed)
  • Fix sub._login parsing
  • Fix error when two bills happen on the same day
  • Id of details are not uniques anymore, so take build a new custom one

Modules: funmooc

  • Base url of the website has changed
  • Fix module and add logged tests

Modules: ganassurances

  • Add pagination to ganassurances/groupama module
  • Handle history and coming of another account type

Modules: github

  • Port to browser2
  • Shorten some long lines + fill url of Issue
  • Few fixes and tests
  • Use CacheMixin new class

Modules: googletranslate

  • Bump to browser2

Modules: groupamaes

  • Fix bug when there is no credit anymore on PEE / fix account typing
  • Handle investment
  • Fix groupamas amount parsing
  • Fix crash when parsing investments
  • Fix portfolio_share range
  • Improve investment management (quantities of values of each funds are divided by availibility date)
  • Fix amount of withdrawal (Sadly withdrawal are not incomes)
  • Add a label in order to detect negative amount in transactions

Modules: happn

  • Fix issues with new kind of ids

Modules: hsbc

  • Fix account type
  • Add more history with pagination
  • Cache accounts and fetch IBANs
  • Fix accounts typing
  • Fix rib navigation
  • Fix currency of accounts
  • Auto delete card summary
  • Handle when website duplicates tr list
  • Raise not error implemented on life insurance for history
  • Add more account types to avoid crash

Modules: ideel

  • New way of handling free shipping (#2146)

Modules: ilmatieteenlaitos

  • Adapted to site changes

Modules: imdb

  • Update imdb favicon to make it more different from the allocine one

Modules: imgur

  • Implement gallery search and CapImage
  • Add date attribute

Modules: ina

  • Fix duration parsing

Modules: indeed

  • Search for words in the entire advert, not only in the title
  • Indeed now uses https

Modules: ing

  • Use CleanDecimal to simplify parsing
  • Add history for life insurance accounts
  • Fix ing 500 error on bill download
  • Fix investment on multiple line
  • Handle blank transaction label
  • Loop on coming from market website to avoid 500 errors
  • Fix missing some investments
  • Increase requests timeout to 60 seconds
  • Fix id of life insurance
  • Fix investment on life inssurence and history on all accounts
  • Fix name of Account._id field
  • Fix navigation on the new life insurance pages
  • Refresh account link
  • Quick fix navigation
  • Ahistory to some asv and more
  • Get IBANs for accounts
  • Transactions detail might be unavailable
  • Some fixes on market accounts
  • Use new CapBankTransfer API
  • Only go on IBAN page for valid account
  • Fix transfer method signature
  • Catch action needed
  • Seems like response headers of download might change
  • Don't raise ActionNeeded for a check reception notice
  • Disable CapTransfer since it's broken
  • New login for asv page
  • SSL errors due to a bad certificate, so create it manually and fix a false https
  • Avoid re-login after lifeinsurance and market pages
  • Before many operations, try to go on main website first

Modules: inrocks

  • Bump to browser2 / inherits from genericnewspaper module

Modules: ipinfodb

  • Bump to browser2

Modules: itella

  • Update to use JSON API
  • Handle missing arrival date
  • Use status from JSON instead of guessing

Modules: jcvelaux

  • Add api_key backend option
  • Fix str-unicode warnings and port to browser2

Modules: jvmalin

  • Add a favicon

Modules: lacentrale

  • Bump to browser2
  • Module connot handle products without criteria
  • Improve products management

Modules: lcl

  • Remove SSL hacks (no need to RC4 anymore)
  • Add individual loans to lcl
  • Fix loans post bourse deconnection, fix loans regex
  • Fix unique id on loans lcl
  • Add certhash
  • Use valuation for BoursePage objects, not balance
  • Raise wronpass if the password is not a digit
  • Change label of transactions and yield card_summary
  • Retry read of amounts (hack to fix broken HTML)
  • Don't return 0 as account ID
  • Fix bad isin like FR0000120404 AC
  • BUMP LCL entreprise browser to bump2
  • Fix isins like FR12345435 ABC
  • Code of investments might be unavailable
  • Rework of transactions type parsing
  • Don't crash when tbody is missing on market accounts table
  • Add a new account type
  • Get more history on several account types
  • Fix matching asv accounts
  • Keep check type on labels
  • Fix life insurance matching from site to site
  • Fix lifeinsurance
  • Check if the IBAN is for the current account
  • Fix crash if there is no IBAN
  • Catch password expired
  • Fix duplicate id on lcl pro
  • Fixing transactions are in multiple account
  • Fix params of history navigation when only one account
  • Add transfer support and use new api
  • Ignore HTTP error 500 on history
  • Have a default accounts_list to be able to bypass login
  • Add transfer.account_balance
  • Fix investment unitvalue
  • Fix iter recipients on loans
  • Handle loan not available
  • Fix lcl espace pro and enterprise login
  • Adding advisor for lcl
  • Add more regexp for transaction types

Modules: ldlc

  • Better way to get subid
  • Add a favicon

Modules: leboncoin

  • Adapt to new website design
  • Fix incomplete url in pagination
  • Website now uses https
  • Fix description field filling
  • Support price_per_meter new attribute
  • Fix typo for poitou charentes region
  • Add url attribute on objects built from list
  • Fix cost for items in rent search page

Modules: lefigaro

  • Bump to browser2 / inherits from genercinewspaper

Modules: liberation

  • Bump to browser2 / inherits from genericnewspaper

Modules: logicimmo

  • Retrieve a bigger photo
  • Do not crash when area is unknown
  • Support price_per_meter new attribute
  • Fix town search

Modules: lutim

  • Fix tests

Modules: mailinator

  • Port to apibrowser

Modules: mangafox

  • Add a favicon

Modules: mangago

  • Add a favicon

Modules: mangareader

  • Add a favicon

Modules: marmiton

  • Always retrieve title

Modules: mediawiki

  • Implement CapImage for searching images
  • Use https by default
  • Fix image search

Modules: minutes20

  • Bump to browser2 / inherits from genericnewspaper

Modules: monster

  • Fix: site changed

Modules: myhabit

  • Update after website has changed (#2615)

Modules: nettokom

  • Add a favicon

Modules: newsfeed

  • Populate url attribute for messages

Modules: okc

  • Revert "do not remove old threads anymore" introduced in 1.1

Modules: oney

  • Fix login
  • Verify SSL/TLS certificate again
  • Raise wrongpass when password is not 10 chars

Modules: opensubtitles

  • Fix opensubtitles small site change

Modules: orange

  • Fix orange module which works with sosh
  • Add label for bills
  • Support multi account and more
  • Remove old url

Modules: ovh

  • Fix login, remove API calls (fixed several times, OVH login page often change)
  • Add a favicon

Modules: pap

  • Remove dots from cost
  • Support new price_per_meter attribute of CapHousing
  • Update to the new website

Modules: pariskiwi

  • Fix module and prefer json mediawiki api

Modules: parolesmania

  • Fix song search
  • Site changed, search artist corrected
  • Adapt module to not use SongLyrics construtor anymore
  • Bump to browser2
  • Artist search: remove page calls to browser methods

Modules: parolesmusique

  • Site changed, search song and artist corrected
  • Adapt module to not use SongLyrics construtor anymore
  • Bump to browser2
  • Artist search: remove page calls to browser methods
  • [parolesmusique] artist search : remove page calls to browser methods, browser now does what it should : browsing

Modules: parolesnet

  • Adapt module to not use SongLyrics construtor anymore
  • Bump to browser2
  • Artist search: remove page calls to browser methods
  • Fix parolesnet, iter artist lyrics : just consider first song list to avoid ID duplication

Modules: pastealacon

  • Fix test_spam test
  • Force site encoding, it is iso8859-15, not utf-8
  • Fix post test when backend is not configured

Modules: paypal

  • Update account_type at the beginning of get_accounts method
  • Fix crash on old website
  • Fix parsing of converted amount
  • Add english word to skipped transactions
  • Fix part converted amount for some transactions
  • Bypass new anti-parsing protection of the website
  • Support multiple accounts with different currencies
  • Get data-ads-challenge-url attribute (anti-scrapping protection again)
  • Catch 500 on detail and skip transaction
  • Fix detection of account currency in weird case
  • Consider that tr is valid if transactionStatus not present
  • Fix currency detection
  • Better detection of currency
  • No crash on securisation step
  • Fix domain of detail navigation
  • Fix transactions detail link
  • Fix parsing of converted amount
  • Fix crash on transaction conversion
  • Skip transactions labeled 'Bill to'
  • Display a warning when a currency is not found and ignore account
  • Fix account_type detection
  • Handle more home page to detect account type
  • Fix paypal payback
  • Handle more login page
  • Add home page paypal
  • Skip fake transactions
  • Handle another wrongpass page
  • Exclude draft transactions
  • Account detection on accountPage too
  • Fix value seperator detection on perso accounts
  • New fixes for login issues
  • Raise unicode message in BrowserIncorrectPassword exception
  • Use json data for transactions parsing
  • Pagination and some improvements
  • Problems with amount and decimals
  • Skip empty transactions
  • Set Iban as NotAvailable for all Paypal accounts
  • Handle english transactionStatus
  • Skip broken transactions
  • Exclude fake transactions (for english website)
  • Skip suspended transactions
  • Handle textual amount is integer
  • Exclude fake transactions
  • On old website, try to go on the new one
  • Skip 'Annulation des frais de PayPal' transactions
  • Paypal commission are now net minus gross
  • Detect ActionNeeded
  • Computation commision if it is not null
  • Add one more login parameter
  • Parse secondary transaction based on label

Modules: piratebay

  • Ported to browser2
  • Handle fuzzy dates

Modules: poivy

  • Add a favicon

Modules: popolemploi

  • Fix search: site changed

Modules: presseurop

  • Bump to browser2 / inherits from genericnewspaper

Modules: prixcarburants

  • Bump to browser2 / handle date of update
  • Fix parsing date (#2606)
  • Adapt module for new comparoob API
  • Website behaviour changed (#2634)
  • Set default configuration for tests
  • Update url to new website

Modules: radiofrance

  • Fix: all websites are updated
  • Fix franceinter
  • Fix france culture podcasts
  • Handle selections and podcast from francetvinfo

Modules: regionsjob

  • Handle case advert is coming from another website
  • Add a place filter

Modules: residentadvisor

  • Add a favicon

Modules: s2e

  • Fetch history on the right account
  • Remove id on transactions: it's not really an id
  • Fix capeasi login page and condition on fetching transactions
  • Using website with more details
  • Fix transactions investments valuation parsing
  • Fix parsing of accounts and history pages
  • Add currency support, add new account types
  • Don't crash when the customer has no account
  • Catch another real no accounts
  • Handle multientreprise
  • Fix transaction amount sum
  • Add new SWISS account type
  • Handle maintenance page
  • Add new hashes for virtual keyboard
  • Detect more no accounts
  • Handling weird wrongpass
  • Add a favicon
  • Add AMF code to investments

Modules: seloger

  • Fix xpath for looking into cities
  • Retrieve all cities
  • Fix typo for photos
  • Do not crash if area is unknown
  • Add support of new price_per_meter field for CapHousing
  • Fix pagination

Modules: senscritique

  • Fix: site got updated
  • Fix: json url changed
  • Fix parsing date

Modules: societegenerale

  • Add cookie before login
  • Fix account type
  • Handling some wrongpass
  • Bump SGPE browser to browser2
  • Add rdate to lifeinsurance for the sake of sorting
  • Do not yield canceled transactions
  • Raise wrongpass instead of crash for non-digit passwords
  • Handling multispace on sogepro
  • New soge entreprise browser
  • Add new type for transaction
  • Add deffered card detection
  • Website limit login to 8 char
  • History pagination
  • Catch login forbidden
  • Fix xpath on invest
  • Handle the two different parsing cases on invest
  • Catch wrongpass
  • Investments are now more detailed
  • Add advisor
  • Switch to browser2
  • Fix pagination in life insurance history and use Form class
  • Add transfer support
  • Fix ix account id being different in sending panel and list
  • Handle renew password and raise ActionNeeded

Modules: sueurdemetal

  • Fix fillobj method

Modules: t411

  • Site url changed
  • Get get_torrent_file() to work
  • Add tests

Modules: taz

  • Bump to browser2 / inherits from genericnewspaper

Modules: transilien

  • Fix: site changed

Modules: twitter

  • Fix: site changed

Modules: ups

  • Use https

Modules: vicsec

Modules: vicseccard

  • Website updated (#2134)

Modules: vimeo

  • datePublished attribute was renamed to uploadDate
  • Handle JSON format changes
  • Fix categories videos in order to use API

Modules: virginradio

  • Correct a bug occuring when the StreamTitle is empty

Modules: wellsfargo

  • Handle webpages with no transaction activity (#2123)
  • Try to get a statement multiple times (#2123)
  • Handle SMS security code, leap and non-leap years (#2618)

Modules: wordreference

  • Fix module for new website
  • Update tests

Modules: yahoo

  • Bump to browser2

Modules: youjizz

  • Fix parsing of video urls
  • Support mp4 videos
  • Use now https, but not for all videos (sometimes not available with https)

Modules: youtube

  • Use googleapi module instead of gdata which is obsolete

Weboob 1.1 is (finally!) out

Posted by Florent Fourcot,

The 1.0 was the first stable release, and one of the stability proof is the time between 1.0 and the 1.1, even with a fast module development.

This release is really huge (1270 commits), and that is very hard to summarize it since there are not really new big features but many of smaller ones. The most important thing is probably a big warning: the 1.1 is the last version with python 2.6 support. We will not try anymore to be compatible with this old version, and we will now focus the work to be compatible with python 3.

To speak numbers, this version was made by 35 contributors. We have 28 new modules for only one new application. We probably reached a majority of website usage with our previous applications. We also removed two modules: grooveshark is down, and hellobank users should now use bnporc module instead (the same code is used for both websites).

As promised, the 1.1 branch does not break the API, except for one thing: the async parameter is now deprecated in the browser and a new name is_async should be used, async becomes a reserved keyword in recent python versions.

Credits

General

  • Last release with python 2.6 support
  • New CapShop capability (#1662)
  • New shopoob application
  • Add new 'weboob' script to find applications
  • New APIBrowser class
  • Add a contrib script to help forking weboob
  • New WEBOOB_DATADIR environment variable

General: removed modules

  • Removed grooveshark module: website is closed
  • remove module hellobank, use bnporc instead

General: new modules

Core

  • Repositories update: display a message when all modules are up-to-date
  • Log full backtrace on module build error
  • Update windows installation (#1668)
  • Better detection of gpgv.exe location
  • Fix issues when using NamedTemporaryFile on windows
  • Add a datadir option, handle WEBOOB_DATADIR
  • Use gpg by default rather than gpgv for signature verification
  • Create empty temporary GPG homedir
  • Exclude backend using instance_name instead of module_name
  • Try to be consistent in variable name between backends and modules
  • Convert StopIteration for PEP 479

Doc

  • Account() constructor does not take an id
  • Fix symlink creation in setup.rst
  • Use browser2 in module documentation
  • Add few docstrings in core and browser

Filters

  • Add Eval filter, for arithmetic expressions
  • Add Slugify filter
  • Add a JSValue filter to look for literals
  • Add Decode filter, to decore urlencoded strings
  • CleanText: add Unicode normalization
  • Regexp: nth param can now be '*' to find all
  • Attr: fix error message
  • CleanText: fix paramter name "children"
  • DateTime: handle TypeError exception raised by parse_date
  • Duration: correct '_regexp' attribut name
  • Dict: several improvements
  • Join: improve it to work like join command
  • Join: add ewMone, addBefpre, addAfter parameters
  • Async: if AsyncLoad parameter is None, do not try to open it
  • CleanDecimal: convert INT, float and long (and not only str)
  • Attr: fix an error message
  • Dict: handle TypeError in parsing
  • Time: Update regexp to catch more values

Browser

  • Add comments on Form class
  • New unit tests
  • New api for content, encoding, and document building
  • Add ChecksumPage class
  • Cleaner way to get rid of BOM for CsvPage
  • Fix wrong encoding detection logic
  • Detect XML encoding
  • Enable passing the path to a CA file to Browser.VERIFY
  • Add an asset method to get absolute path of module local files
  • Browser.VERIFY can now be a module local file
  • Fix compatibility with python 2.6 in HTMLPage (#1651)
  • TableElement: fix count columns when there is an attribute "colspan"
  • Do not display urllib3 warning if there is no TLS certificate verification
  • Add an IPhone profile
  • Add a GoogleBot profile
  • Add method Browser.set_profile
  • DomainBrowser: ability to give the baseurl in constructor
  • Add LoginBrowser.do_logout() method
  • Handle "Refresh" meta tag in HTMLPage, with a REFRESH_MAX option
  • Add an absurl method to Page
  • Fix crash when using HEAD on a page-matching URL
  • Add BrowserQuestion exception (ask user to change config when raised)
  • Add 'replace' xpath function in HTMLPage
  • Introduce new DictElement class
  • Add some JSON helpers
  • Strip data on CsvPage
  • Better control of HTTP referrer
  • Allow changing the Session class
  • Pass async/sync callbacks in the same way
  • Bump Firefox version to next ESR (38)
  • Fix documentation: use HTMLPage instead of Page
  • Add StatesMixin class to store state of a browser
  • Change worker pool size after session init
  • Allow to force method in URL.go (pass argument to location())
  • Support using id in HTMLPage.get_form()
  • Rename async parameter to is_async

Old Browser

  • Do not use PROTOCOL_SSLv3 as it is deprecated, use bad named
    PROTOCOL_SSLv23 instead
  • Add BaseBrowser.get_page method (don't change the location)
  • Add a StateBrowser to store cookies in storage
  • CSVParser: catch exceptions
  • Disable lowsslcheck if there is a proxy, as ssl doesn't deal with it

Capabilities

  • Add the possibility to define Field type argument as a string
  • Create enum. Use it in calendar and housing (code factorization)
  • Add a way to instantiate an object from a dict of its fields values
  • Add russian ruble and XOF currencies
  • Add SGD, BRL, JPY and MXN in currencies

Capabilities: Audio

  • Move specific audio filters from capabilities/audio.py to tools/capabilities/audio/audio.py
  • Add specific filters to create audio objects id
  • Do not require an id in constructors

Capabilities: Bank

  • Fix StatusField to be a BaseObject
  • Change quantity type to decimal number
  • Better explanation for Investment class
  • Introduce new type Account.TYPE_LIKE_INSURANCE
  • Add credit limit, payment due date and amount for credit card bank
    accounts (#1717)
  • Add IBAN properties
  • Add card number and commission in Transaction
  • Add original_amount and original_currency in Transaction
  • Add country in Transaction
  • Add Transaction.investments and Investment.vdate attributes
  • Add Account.valuation_diff attribute
  • Add Account.number attribute

Capabilities: Bill

  • Inherits Detail and Bill from Currency
  • Remove Bill.idparent attribute
  • Add Bill.vat attribute

Capabilities: Calendar

  • Add timezone parameter to calendar event in order order do get the good date/hours in ical exports
  • Add a field 'summary' to Query object
  • Add ticket status in BaseCalendarEvent
  • Add a ticket field to Query
  • Add BaseCalendarEvent.timezone attribute
  • Add new categories

Capabilities: Cinema

  • A person may have now several roles in a movie

Capabilities: Collection

  • Do not require an params in constructor

Capabilities: Contact

  • Set default value of Contact.profile to OrderedDict()

Capabilities: Dating

  • Fix API to store optimization name in object ID
  • Correct method name: s/search_gallery/search_galleries/

Capabilities: GeolocIp

  • Add osmlink field

Capabilities: Housing

  • Support new type SHARING

Capabilities: Messages

  • Remove _Message and _Thread and use strings in Field instead

Capabilities: Recipe

  • Do not force to use id in object constructor

Capabilities: Torrent

  • Remove name mandatory parameter in constructor

Capabilities: Thumbnail

  • Thumbnail now inherits of _BaseImage

Capabilities: Weather

  • Do not require an id in constructor
  • Fix default value for date
  • Cosmetics in temperature formatting

Applications

  • Backport check_output function for python 2.6
  • Remove wconio dependency on win32 platform
  • Allow ResultsCondition to compare timedelta
  • Encode properly output in console.py (#1673)
  • Fix BrowserSSLError catch in ConsoleApplication (#1702)
  • Handle attribute error when stdout does not habe encoding attribute
  • If backend name is already taken, suggest the first available one with
    the same base
  • Formatters: use weboob.tools.misc.guess_encoding instead of utf-8
  • Formatters: manage tty colummns while displaying text
  • Use os.devnull and binary flag for windows compatibility

Applications: REPL

  • Fix a crash when ReplApplication.do() is called with a function
  • Create a message for 403 errors

Applications: boobank

  • Format: ISIN is a 12 letter word
  • Format integer for quantity
  • Sometines the full code is not available. Display description instead
  • Add a command 'Budgea' to export accounts and transactions
  • Fix investment display when some fields are empty
  • Identify credit card for each transaction in OFX export

Applications: boobill

  • Use do_ls to cache objects in REPL
  • Add binary mode for writing. Fix file downloading on windows

Applications: boobcoming

  • Fix crash without end_date (#1667)
  • Add search by summary
  • Print ticket status
  • Add search by ticket status
  • Handle error correctly in do_attends
  • Add a load command
  • Improve ical export (end lines with '\r\n', as RFC says)
  • Add timezone in ical export
  • Display price as float
  • Improve date display
  • Do not crash during export of an unknown event

Applications: cineoob

  • Give roles informations in persons_in_common and movies_in_common
  • Support several roles in the same movie for an actor
  • Movies in common are now sorted by release year
  • Fix problem with stream parameter in request.get in python-requests v2.5
    (#1683)

Applications: flatboob

  • Display backend in city choice

Applications: geolooc

  • Fill OpenStreetMap link

Applications: havedate

  • Store optimization name in object ID

Applications: parceloob

  • Clean cached objects before getting new ones

Applications: qcineoob

  • Add 'view' and 'view in new tab' buttons for all search results
  • Add 'view thumbnail' for movie and person search results
  • Factorisation of events in search results
  • Persons in common implemented in movie page
  • Movies in common implemented in person page
  • Support several roles in the same movie for an actor
  • Better display of search results, spaces added
  • Increase biography field size
  • Movies in common are now sorted by release year
  • Better user input validation for same-person movies in common

Applications: qcookboob

  • Implement tab navigation (#1290)
  • Fix bug when thumbnail_url is empty

Applications: radioob

  • Can now download full album
  • Add file in search subcommand

Applications: videoob

  • Fix the problem with streamed calls on requests (#1683)
  • Display download subcommand in debug mode

Applications: weboorrent

  • Fix formatter for info command

Applications: weboob-config

  • Improve info command to allow json formatting
  • Put all config details in a dictionnary
  • Format capabilities as JSON array
  • Add installation status in modules subcommand
  • Prompt user to accept an untrusted keyring (#771)
  • Add module_name param in add_backend to allow command line interaction
  • Update help of "add" command

Tools

  • Move guess_encoding into weboob.tools.misc
  • Add a Javascript class
  • Avoid crash on UnicodeDecodeError with retry decorator

Tools: bash-completion

Tools: date

  • Supporting french month abbreviation without dot
  • Extend date_max_bump default value to 31 days in LinearDateGuesser and
    ChaoticDateGuesser
  • Add some french translation
  • Parse french date with DMY order by default

Tools: virtkeyboard

  • Fix not logical arguments order
  • Dump of tiles: save full image too
  • Allow to receive a list in get_symbol_code() method
  • Support pollution in coords attributes

Contrib

  • Add a gtk appindicator for boobank
  • Remove bashisms in report_accounts.sh
  • weboob-generic: fix object listing when tomonitore is not set

Contrib: downloadboob

  • Print link message only when needed
  • Fix new file detection
  • Support download of m3u8 files
  • Allow passing config and section filter on command line
  • Replace call to 'fill_video' by 'fillobj'

Modules: 750g

  • 750g is now working using browser2 (#1706)
  • Fix: Only select the first thumbnail image
  • Fix: Site changed
  • Improve instructions parsing

Modules: adecco

  • Bump to browser2

Modules: agendadulibre

  • Add missing favicons
  • Adapt for timezone support
  • Do not crash when json included in webpage is not valid
  • Correctly retrieve start and end date

Modules: allocine

  • Remove excessive quote_plus call on pattern to make multi-word search work again
  • Adapt for more informations in persons_in_common and movies_in_common
  • Fix: accents were causing problems in search
  • Handle videos from allocine
  • Handles showtimelist in allocine
  • Fix: bug while playing videos from external url
  • Fix: syntax error on allocine
  • Adapt for timezone support
  • Max rallocine rate is 5
  • Fix unicode warning

Modules: allrecipes

  • Adapt to new version of website
  • bump to browser2

Modules: amazon

  • Skip "In transit" orders. (#1684)
  • New discount: "Lightning Deal" (#1684)
  • Generate payment transaction if there aren't any (#1684)
  • Order charges xpath fix (#1684)
  • Amazon module: skip 'Not yet shipped' orders (#1699)
  • Amazon module is updated according to the website change (#1715)
  • Skip "On the way" orders at Amazon (#1716)
  • Support new version of Amazon order details header (#1721)
  • Retry Amazon requests after ReadTimeout exception
  • handle Timeout exception as ReadTimeout has been introduced in recent versions of requests
  • Add French Support and Shopoob compatibilities
  • Add french translations
  • Retry ConnectionError as well (#1742)
  • pep8 fixes
  • Support for a new payment layout and priceless items (#1764)
  • Scrape the updated website (#1808)
  • Update after site changes
  • New status to ignore
  • More shippings and discounts (#1853)
  • Retry getting an order page (#1876)
  • amazon bills
  • Fix: work with iter_resource, and use ISO currency codes instead of symbols

Modules: ameli

  • Update after sites changes, and upgrade to new browser
  • Handle authentication errors
  • PaymentDetailsPage: use a simpler URL
  • Site was updated
  • details isn't implemented: remove dummy code
  • ameli: prevents crashes when no bills are available

Modules: amelipro

  • Update after sites changes, and upgrade to new browser
  • Fix decimal bug

Modules: americanexpress

  • Handling no amount on account
  • Add default date to beginning_debit_date
  • Handle new americanexpress site but only if there is only one card
  • Increase timedelta to guess real date
  • Handle US account
  • Amex is dumber and dumber ( increase time delta because amex have hard time to put the date in a good interval)
  • Fix parsing of rdate
  • Fixing card type
  • Removing useless id

Modules: apec

  • Use https instead of http
  • Adapt to the new version of the website and use browser2

Modules: arretsurimages

  • Fix: get videos titles in h1 (#1447)
  • Fix: correctly implement search and add a test (#1691)

Modules: arte

  • Fix bug id was not well filled in info command and title was not filled
  • Adapt to browser2
  • Fix display bug during newly install
  • Default value must be key, not label
  • Fix bug when urls are not found
  • Fix bug when date is not available
  • Handle special characters in search
  • Fix arte concert (a fields has been removed fron json)
  • Handle special cases in json parsing
  • Do not raise 404 error while searching a video using an other site url
  • Fix program menu management
  • Fill id using a better value for arte cinema

Modules: audioaddict

  • Fix description data removal from audioaddict API
  • Rename SKY FM to Radio Tunes
  • Add Fresca Radio

Modules: aum

  • Search query string: replace %(lat)s and %(lng)s with position in my profile
  • Fix parsing of geo position

Modules: axabanque

  • New login pages on axabanques (#1709, #1711)
  • account.id is now only paramNumCompte if there is not paramNumContrat
  • Handle expired password
  • Handling banned account
  • Do not try to recover transaction from DAT account
  • axabanque fix duplicate accounts id
  • Handling duplicate id for account using Jsp Id
  • Stop parsing history of investment accounts
  • Fixing random number formating
  • Add error page

Modules: banqueaccord

  • Fix inconsistent history on loan accounts if there is no init date
  • Labels of login fields changed

Modules: banquepopulaire

  • Support new authentication on some region websites
  • Fix login on new website (needs upper case login)
  • Fix crash for new cards with no debited history
  • Break the new protection
  • Support full list of accounts (#1360)
  • Fix crash in a particular case of navigation
  • Supporting new account page
  • Parsing last connection date correctly
  • Fix navigation when passable unavailable page is displayed before login
  • Support case where user may accept TERMS_OF_USE
  • No history if account has no prev_balance
  • Apply alsace and lorrainechampagne fusion in bpalc in banquepopulaire module
  • Add Investment accounts
  • Correctly return an iterator even if this isn't an investment account
  • Fix crash on empty field banquepopulaire investment
  • Add life insurance investment
  • Skip message pages to get_investment on banquepopulaire
  • Fix a crash on iter_accounts after an investment on banquepopulaire
  • Remove stored 'token' attribute from browser and fix navigation on banquepopulaire website
  • Correctly get token only when available
  • Handle error page to avoid global crashes
  • Fix to get token
  • Adding support for IBAN
  • Fix getting tokens on some pages
  • Attempt to fix cgu validation
  • Adding loan
  • Add unavailable page
  • Fixing useless id
  • Ignore \0 bytes in documents

Modules: barclays

  • Add AssurancePage on which we don't support get_history
  • Fix barclays navigation
  • Handle suspended accounts
  • Fix bug when card is first account
  • Support more account types
  • Sorting transaction before returning
  • Fixing page payement to foreign currency
  • Remove print
  • Remove bad transactions ids

Modules: biplan

  • Site changed
  • Detect invalid events
  • Do not crash when no time can be found
  • Add timezone / fix encoding and hours detection
  • Fix parsing error on start date

Modules: bnporc

  • Fix to get transactions when they are ordered by type
  • Get IBAN of accounts
  • Do not try to get IBAN for delegated accounts
  • Add investments to bnporc module
  • Handle corner case
  • Move old code for old site in a new folder
  • Add support for new bnp site
  • Move new browser into pp/
  • Parse transaction labels, and fix account's coming
  • bnp history not supported on life insurance
  • Investment not implemented for entreprise browser
  • bnp investment fix
  • Handle error message after login
  • bnp type life insurance on capitalisation accounts
  • Do not get investments from PEA Espèces
  • Handling account without history
  • bnp history always return an iterator
  • New bnp entreprise certhash
  • Switch to new website and several fixes
  • Correctly set IDs
  • Raise BrowserIncorrectPassword when user should use hellobank instead
  • Raise IncorrectPassword after 100 connections
  • WIP on new company bnp website
  • Adapt module to use Browser2
  • Add company browser
  • Fall back on old website for pro accounts
  • Add iban field for company accounts
  • New certificate
  • Fix compatibility with python2.6
  • Fix concerning banqueprivee subdomain
  • Handle website unavailable for history
  • Better handle of expired password
  • rename pp folder to ppold
  • Fix PEP8 E713
  • Support lifeinsurances on new website
  • Add authentification error code
  • Skip transaction without date
  • Adding support iban for some pro account
  • Automatically update password to the same one when the 100 connections threshold is reached
  • Type more accounts to get history and transactions investments on those
  • Revert fall back on oldwebsite for pro accounts
  • Add a page to renew password
  • Support checking type
  • Adding loan for crédit immobilier
  • Calculate IBAN key if messing
  • Add Hellobank support
  • Use shorter id
  • Revert "[bnporc] use shorter id"
  • Handle investment and history on market accounts
  • Fix crash if bnp throws 500 HTTP status code
  • New connection threshold page url for hellobank
  • Valution_diff on market accounts
  • Add number to iter_accounts
  • Fixing bad id for coming transaction
  • Prevent us from crashing when bnp crashes

Modules: boursorama

  • Add investment
  • Support market investments
  • Fix duplicate entry on card account
  • Cleaner way to iterate
  • Better explanation for Investment class
  • Fix website changes (virtual keyboard, accounts list) (#1696)
  • Ignore moneycenter accounts
  • Minor fix
  • Introduce new type Account.TYPE_LIKE_INSURANCE
  • Use two_authentication with BrowserToBeContinued instead of raw_input
  • Use StateBrowser, s/BrowserToBeContinued/BrowserQuestion/ and coding style fixes
  • Remove text after card number if any
  • Raising incorrecte password when profil is incomplete
  • Fix CC history for boursorama
  • Update hash of boursorama's certificate
  • Fix duplicate accounts
  • bourso new investment page and fix parsing
  • Fix crash when there is more data in the cell
  • Fix PEP8 E713
  • Fix parsing of unit price
  • Fix investment link parsing
  • Ignore immediate debit cards
  • Investments on lifeinsurance transactions
  • Fix pagination of lifeinsurance history
  • Ignore promotional fake accounts
  • Fix parsing some special market accounts
  • Valuation diff on lifeinsurance accounts
  • Fixing bad id transactions
  • Do not use useless id
  • Fixing false id in transactions
  • Fix parsing of investment label
  • Fall back on old website if we are on the new one

Modules: bouygues

  • Rewrite backend using browser2 (#1970 and #1978)
  • Add support for bill on bouygues
  • Avoid useless login
  • Handle website modifications
  • Home page uses https (#2510)

Modules: bp

  • Support professional accounts
  • Update upstream certificate hash (#1702)
  • Handling when website have an error
  • Fix parsing of virtual keyboard url
  • Add a page to pro website
  • Catch a no online banking service at login
  • Set type of accounts on professional accounts
  • Identify credit card for each transaction
  • Update to new certificate
  • Adding IBAN support for banque postal pro
  • Add a regexp on withdrawal labels
  • Parse IBAN for personal accounts
  • Do not crash if there is no IBAN for an account
  • Iban in unicode
  • Fix for newer pdfminer versions
  • Fixing bad account type on pro website
  • Sorting transactions before returning them
  • Fix for new API
  • Only call initialize if needed
  • Fix get_pages usage for new API
  • Fix newapi import
  • Fixing fake id for transactions
  • Fixing pdf parse with new api
  • Fixing no RIB

Modules: bred

  • Support new website of bred
  • Fix compatibility with Decimal of python2.6 (#1685)
  • The 'categorie' key isn't always in json
  • Detection of disabled accounts
  • Correctly strip currency code
  • Correctly deal with cookies storage
  • Add type Account.TYPE_DEPOSIT
  • Catch technical error at login
  • Fix duplicates accounts
  • Fix PEP8 E713
  • Get IBAN on accounts
  • Dd certash
  • Handle multiple universes
  • Oops fix mistake

Modules: caissedepargne

  • Fix getting card accounts on old website
  • Set type on card transactions
  • Support Banque BCP which uses same website than caissedepargne
  • Do not force TLSv1 anymore as in some cases it doesn't work
  • Investment on lifeinsurance and market account
  • Add balance on NUANCE
  • Fix IndexError related to investment feature
  • Handling website error for market place
  • Fixing index error when checking if we are on error page
  • Fix BrowserUnavailable when browser is on market pages
  • Iban on cassedepargne accounts
  • Add new certhash
  • Ignoring CB transaction aggregation
  • Try to type more accounts
  • Adding support for saving type
  • Prevent crash on market
  • Get more investments
  • Valuation_diff on market accounts
  • Fix crash if there are several market accounts
  • Fix not authorized to read market accounts
  • Fix accounts parsing on banquedelareunion
  • Adding compte titre type
  • Fixing leaving investment space

Modules: canalplus

  • Correctly fix the problem with streamed calls on requests (#1683)

Modules: carrefourbanque

  • Fix accounts parsing
  • Fix 404 on history and now support "compte livret"
  • Fix when account balance is negative

Modules: cci

  • Update for Join filter changes
  • Site changes

Modules: chronopost

  • Status guessing

Modules: cic

  • Handle new certificate
  • Supporting mouvements.html and situation_financiere.html
  • Fix list index out of range on iter_accounts
  • Rewrite module to browser2 (based on creditmutuel)
  • Cic can have letter in login
  • Add IBAN on cic
  • Fixing cic account with "Contre-valeur"

Modules: citelis

  • Certificate hash change

Modules: citibank

  • Do not wait for transactions if there's none. (#1661)
  • Credit limit, payment due date and amount for Citibank. (#1719)
  • Use V8 for login (#1743)
  • Relogin when the website returns garbage instead of a PDF. (#1820)
  • Site was updated. Closes (#1820)
  • Parse thousands correctly in the statements. (#1984)
  • Backport check_output for 2.6

Modules: cmb

  • Fix detection of authentication fail
  • Add deposit type in CMB module
  • Fixing when login is obsolete
  • Fixing cmb with some pro module
  • Quick fix for multiple space.

Modules: cmso

  • Add pro browser to cmso bank module
  • Correctly raise instance of BrowserIncorrectPassword
  • Correctly handle 500 error during login
  • Fix parsing of market accounts and factorization of the Transaction class
  • Fix website changes on auth page
  • Go to subscription page before get accounts for some users
  • Use a local instance of LinearDateGuesser instead of global one
  • Support pagination of transactions
  • Fix: coma are used as decimal separators
  • Correctly handle expired sessions
  • Fixing duplicate entry
  • Handle double site type on the pro side
  • Clean + blackbox compatibility on cmso pro

Modules: colisprive

  • Rewrite for browser2
  • Fix history parsing

Modules: cragr

  • Fix parsing of labels with new crédit agricole website
  • Fix parsing of raw transaction text
  • Handle useless pages to prevent being redirected to mobile version
  • Get IBAN on accounts
  • Detect connection errors
  • Add new labels to market type
  • Fix account list parsing, skip irrelevant account
  • Fix getting history for card accounts
  • Fix import of Account
  • Add tdbgestion page
  • The type of PEL accounts is now SAVINGS instead of MARKET.
  • Refactor
  • Support life insurance accounts
  • Handle sessionSAG parameter
  • Support market accounts
  • Handling bad investement parsing
  • Handling bad value for investement
  • Fix parsing on cragr investment
  • Fix av investment parsing
  • Do not return NotAvailable if value == 0.0
  • Support loan accounts
  • Fix double clean of investment amounts
  • Add account types
  • DAV PEA is a checking account
  • Handle new login navigation on some sub websites
  • Fix incorrect password on new login
  • Add page to fix login access
  • Fixing categorization
  • Cragr perimeters
  • Adding new saving account name
  • Fix infinite loop when a crash occurs on perimters
  • Fix when some perimeters are crashing
  • Fix broken perimeters
  • Fix in case of CGU poping when changing perimeters
  • Fix card accounts balance parsing
  • Fix mistake

Modules: creditcooperatif

  • Update regexp to support SEPA transfers
  • Fix exception in some cases
  • Fix regexp for external transfer
  • Handle investment
  • Correctly return an iterator even if this isn't an investment account
  • Add attribute _inv to pro accounts

Modules: creditdunord

  • Handle the case when the creditdunord CB label is "OPERATION SUR CARTE" (#1712)
  • Remove spaces from IDs
  • Handling account with no history
  • Handling no details for this account
  • Fix crash on accounts which doesn't support history
  • Handle duplicate accounts
  • courtois: we now skip transactions without date even on pro accounts
  • Fixing wrong id when no details for an account
  • Trying to avoid duplicate accounts
  • Fix courtois we now always return iter for investment
  • Handle IBAN
  • Fix auth fail url
  • Add VirtualKeybord auth
  • Change lifeinsurance accounts type
  • Removing false id for transactions
  • Fix life insurance accounts and add valuation_diff
  • Raise BrowserIncorrectPassword is password isn't digits
  • Fixing history for account
  • Fixing pro account
  • Fix assert for password
  • Avoids infinite loop in case of fail login

Modules: creditmutuel

  • Handle new useless pages
  • Updat transfer method
  • Save and load state of browser into storage
  • Handle refresh on empty page
  • Handling already logged
  • Handling html page
  • /cmmabn/fr/ is a valid login page
  • If do_login() is called, force go on the login page
  • Handle refresh on LoginPage too
  • Handle disconnection
  • Fix accounts parsing
  • Fix parsing of 'soft' parts of account names
  • Fix parsing of balances if accounts page has been configured by user to display balance in several currencies
  • Clear cookies before to login
  • Better form detection
  • Increase timeout
  • Clean transactions label
  • Parse contactless payments ("PSC") transactions
  • Handling orignal currency
  • Using TableCell to handle dynamic transaction list
  • Ignore ELEVE CARTE line
  • Add account types patterns
  • Add type pattern for young accounts
  • Investment for market accounts
  • Fix crash when trying to find reale amount of market accounts
  • Fix portefeuille account parsing
  • Add IBAN on creditmutuel accounts
  • Fix parsing market accounts
  • Fix parsing of unitprice
  • Fix skipping market summary fake account
  • Fix parsing valuation of a market account
  • get more iban
  • Fix parsing isin codes in special cases
  • Handle new website
  • Redirect page added
  • Fix label
  • Fix parsing label of accounts on old websites
  • Fix wrong valuation on market accounts if several are present
  • Valuation_diff on market accounts and new balance for PEA
  • Account typing
  • Fix blackbox compatibility
  • Fetch more transactions
  • Fix detection if transactions are coming or not
  • Fix when last_debit is None
  • New accounts type
  • Life insurance on old website
  • Fill the rdate field on life insurances history

Modules: cuisineaz

  • Site changed / rewritten using browser 2

Modules: dailymotion

  • Fix dailymotion "search" return bug message
  • Fix to get video url

Modules: delubac

  • Rewrite module with browser2 on new website
  • Handle accounts and transactions
  • Website require a digit password

Modules: ovh

  • Adding bill for ovh

Modules: edf

  • Correctly set amount and currency on bills

Modules: ehentai

  • original_title does not exist anymore
  • There is no more description available
  • Update cardinality field name
  • Use Thumbnail object

Modules: feedly

  • Favicon transparency and remove JPEG artifacts
  • Use new DictElement object
  • Improve tests
  • Iimprove login management
  • Fix login
  • Do not crash when there is no title in article

Modules: fortuneo

  • Detection of accounts type
  • Support investments detail
  • Add page to browser: pea account type
  • Introduce new type Account.TYPE_LIKE_INSURANCE
  • Update changed certificate
  • Fix fortuneo module now support "compte especes"
  • Fix support of PEA accounts
  • Fixing empty performance
  • Fixing 2nd detail line who was considered like investment
  • Fixing bad int values on investment
  • Fetch history of life insurances
  • compte-espaces is a AccountHistory page
  • Correctly set type of market accounts
  • Fix savings account type
  • Fixing closed life insurance account on fortuneo
  • Raise IncorrectPassword when there is new id
  • Throwing error when an sms is required to access information
  • Switching error to browser incorrect password
  • Fixing listing account
  • Fix crash when parsing accounts without balance
  • Fix navigation
  • Support deferred debit cards
  • Set type of card transactions
  • Fix new pass required detection

Modules: francetelevisions

  • Fix "search" always return empty list
  • Correctly fix the problem with streamed calls on requests (#1683)
  • Remove "latest" feature
  • Fix pluzz's search (1700)
  • Add option to get video from an url
  • Handle lastest replay videos
  • Fix bug search date in bad balise
  • Fix bug: we can only search one time
  • Use new DictElement object
  • Fix: site changed (#1855)
  • Fix: site changed again (#1868)
  • Update tests
  • Fix bug while getting video from some url
  • Fix error when video url is none
  • Handle url from francetvinfo website

Modules: freemobile

  • Correctly handle wrong credentials
  • Fix parsing of bills urls
  • Force UTF-8 encoding with the standard mechanism
  • Enable bills filtering for multi-account
  • Fix subscription date parsing for multi-account
  • Fix forfait name parsing for multi-account

Modules: ganassurances

  • Rewrite ganassurance with browser2 and fix auth and history
  • Add missing date column title for cards history
  • Fixing statefull website with account's link changing
  • Fixing coming date with an other label
  • Change AccountsPage xpath to only get personal finances accounts
  • SSL is not supported anymore
  • Add changing pass/id page

Modules: googletranslate

  • Fix code used by Google Translate for Greek

Modules: groupamaes

  • Use a local instance of LinearDateGuesser instead of global one
  • Should now always find accounts
  • Fix : site changed

Modules: hellobank

  • Fix permissions, encoding and copyrights
  • Do not try to get IBAN from life insurance accounts
  • Do not try to get history of life insurance accounts
  • Allow transaction parsing across multiple pages
  • remove module, use bnporc instead

Modules: hsbc

  • Handling when website is gone away
  • Fixing shared accounts_list
  • Fix new list of accounts and new URLs
  • No history on loan accounts
  • Handling duplicate account id (#2114)
  • Submit two useless form to mimic browser
  • Fix date parsing
  • Add coming for cards
  • Do not use fake id for transactions from hsbc
  • Allow hsbc user to have more than 8 length password
  • Fixing credit is positive on some account

Modules: hybride

  • Fix: site changed
  • Fix: Date parsing
  • Fix: site changed again
  • Fix bug when there is no end date
  • Iimprove end_date management
  • Update to support timezone
  • Use Decode filter to get the good id
  • Simplify id

Modules: imdb

  • Fix: python datetime is locale-dependent
  • Fix: 'N/A' value in movie release date

Modules: imgur

  • Port to browser2
  • Implement CapGallery

Modules: ina

  • Bump to browser2
  • Handle CapAudio
  • Find video from url

Modules: ing

  • Coding style
  • Change selection of keyboard (more generic)
  • Fix navigation after a transfer
  • Remove hack to convert decimal to int
  • Fix crash if there is no ':' in IDs
  • Update regexp
  • Allow birthday to be entered with separators
  • Support of ASV accounts
  • Fix switch between market and checking websites
  • ASV accounts don't have history
  • Fix detection of errors in page
  • Get valuation information in several locations
  • NotImplemented exception for history on asv and clean code
  • As ING website sucks, re-login to retry getting investments list
  • Introduce new type Account.TYPE_LIKE_INSURANCE
  • Fix website change
  • Be sure we are logout before login
  • Store "where" in ing browser
  • Fix bad login with cookie cache
  • Fix parsing of euro founds
  • Add a kind of life insurance account name
  • Fix parsing of ISIN code for market accounts
  • Fix switches between market and bank websites
  • Euro funds have diff equal to 0
  • Fixing not avalaible value
  • Handling netissima page
  • Fix set of Investment.unitprice for market accounts
  • Correctly get subscriptions even if previous page was on market
  • Fix finding jid on some pages
  • Load ISIN code from details page if not available in the list
  • Fixing empty page in some case in investment without pl in URL
  • Prevent useless request since there is no history on asv accounts
  • Handling +infinity (best investment ever)
  • Fix ISIN code on asv
  • Do not slugify investments without ISIN codes
  • Fix ing bills
  • Parse bills dates
  • Skip summary bills
  • Fix isin on US investment
  • Fix when valuation is not available

Modules: inrocks

  • Url changed
  • Fix url matching
  • Do not try to parse premium news pages

Modules: kickass

  • New domain name
  • Adaptation to new browser, iter_torrents mysteriously not working
  • kickass now works with new browser

Modules: lcl

  • Website change, rewrite with browser2
  • Ignore summary lines in history tables
  • Fix stop parsing not transactions lines
  • Certificate updated
  • Fetch detail of transactions and use it in Transaction.raw
  • Change timeout to 30 seconds
  • Call default Browser.deinit()
  • Do not change Transaction.raw if there is no detail
  • Adding pagination support for lcl
  • Fix async bug with cards
  • Fixing RC4 and cert for pp browser
  • New certificate for LCL Espace Pro
  • Browser doesn't inherit 'object', so 'super' doesn't work
  • Handling account can't use online service when login
  • Fixing bad login when session expire
  • New transaction detail page
  • New certificate for lcl entreprise
  • Fix date parsing bug on transaction label 0/07/2015
  • Wrongpass instead of BrowserUnavailable
  • Fix PEP8 E713
  • Fix contracts selection routing
  • Remove validate method
  • Handle one case of bad login/pass
  • Handle investment
  • Fix for clients with no market investment accounts
  • Fix crash on asv accounts
  • Fix parsing of life insurance accounts with special chars in owner name
  • Fixing when we have some additional information
  • Disconnecting from bourse portal before returning the account list
  • Fix bugs on av accounts, double ids and disc from detail
  • Fix mistake in disc page
  • Fix life insurance accounts ids
  • Fix parsing of market accounts
  • Fix crash on enterprise websites
  • Adding iban info when available
  • Fix pro accounts iban
  • No permission page to fix crash
  • Fix crash in case life insurances are unavailables
  • Support a new account type
  • Fix parsing of some transactions on pro accounts
  • Adding saving account
  • Handling IBAN single account
  • Adding checking account
  • Fixing payement wording
  • Adding SAVING type for account
  • Handle home page so the module is blackbox compatible
  • Adding account type
  • Change page to handle specific behavior

Modules: leboncoin

  • Add missing favicons
  • Fix bug while parsing date
  • Fix bug when area is unknown
  • Add regions support
  • Fix error when there is no price and improve description display
  • Improve cities display in search
  • Fix search city using zipcode
  • Fix bug when user do not choose any city
  • Do not crash if a select does not exist in the form
  • Fix title parsing in search result
  • Get main photo when there is no carousel
  • Fix bug in costs retricton for rent type
  • Fix bug due to other backends cities
  • Fix site changed
  • Fix regex in id detection

Modules: lutim

  • Upgrade to Browser2

Modules: mailinator

  • Fix module as the api changed

Modules: mareeinfo

  • Add missing favicons

Modules: marmiton

  • marmiton is now working using browser2 (#1706)
  • Fix bugs in marmiton
  • xpath change (#2117)

Modules: mediawiki

  • Update tests to use "Project:Sandbox" page
  • Port to browser2
  • Remove useless stuff

Modules: meteofrance

  • Adapt to browser2
  • Use new DictElement object
  • Allow to iter forecasts using town name in addition to city id
  • Site changed

Modules: monster

  • Fix and adapt to browser2
  • Improve page parsing
  • Adapt for new Join filter
  • Handle adverts coming from partner websites

Modules: okc

  • Correctly detect when login fails
  • Handle logged page
  • Make all attributes of optim start with an underscore
  • Fix parsing of profiles
  • Way used to vote has changed
  • Fix encoding of urls
  • Fix parsing of thread title
  • Rewrite okc module with browser2 and less ugly code
  • Skip empty essay fields
  • Add status and orientation fields
  • Do not remove old threads anymore
  • Iter_unread_messages: do not visite read threads

Modules: oney

  • Add new hashes for virtkeyboard
  • Change get_balance xpath
  • Adding hash for virtkeyboard
  • Handle multiple cards

Modules: orange

  • Adapt to website changes (#1828)
  • Add https urls
  • Update maintainer
  • Supporting bill capability for orange

Modules: pap

  • Adapt to new DictElement object
  • Fix bug in parsing date
  • Fix : site changed

Modules: pastealacon

  • Fix: url changed
  • Works with other pastebin sites
  • Revert "pastealacon: work with other pastebin sites"

Modules: paypal

  • Handling when user can't acces account balance on main account
  • Supporting new paypal website
  • Fixing using new account page version when we are already on old account page
  • Handling when currency is not in right place
  • Fixing primary account display twice
  • Fixing wrong id on account
  • Checking website version after login
  • Fix crash on new accounts list (#1654)
  • Fix selection of table containing last file requests to be less ambiguous
  • When balance is zero, paypal add a class to the div
  • Handling new website account for individual
  • Paypal fix for KeyError: 'counterparty'
  • We now ignore transactions with no 'displayAmount'
  • Fix name of paypal accounts (display currency instead of balance)
  • Take care of transaction's net amounts (#1694)
  • Fix warning with adding a new home page
  • Handling home page on new private website
  • Fix paypal new activity URL
  • Fix paypal
  • Use locale.format to format amount
  • Do not use locale.setlocale to parse amount
  • Fix history on paypal, stop parsing suspended recurring payment
  • Get converted amount for foreign currencies
  • Fix crash IndexError: list index out of range (#1747)
  • Remove unused code, fix timeout of history fetch
  • We don't try to parse transactions without currencyCode anymore
  • New cert + fix conversion warning
  • Handling useless captcha page
  • New paypal certificate
  • Execute javascript convert() function on token
  • Paypal handle bad login/password
  • Add paypal new page to fix crash on history of some accounts
  • Fix crash on old website and inform users
  • Change detection of pro or perso account_type
  • Increase DEFAULT_TIMEOUT for accounts with a lot of transactions with foreign currency
  • Fix call of ads_token convert function
  • Fix timeout on accounts with huge transactions number
  • Fixing timeout for some accounts
  • Handling timeout when fetching old history
  • Fix transactions parsing
  • Get real amount accounting paypal's commissions
  • Fixing captcha
  • Handling promo page for new paypal application
  • Rewrite paypal with browser2 and fix login
  • Add a way to detect login errors
  • Pro accounts transactions amount is now returned without commission
  • Ignoring refound offer for paypal
  • Fixing paypal auth challenge
  • Do not supply floats to Decimal constructor
  • Fix login
  • Detect a login error
  • Fix parsing accounts without available balance
  • All personal accounts seems to be on new api.
  • Parse more transactions not in account currency
  • Try to get the activity 3 times
  • Discard another transaction type
  • Fix regexp parsing converted amount of transactions
  • Fix mistake of converted amount parsing
  • Handling payement from another bank
  • Fix parse_transaction return
  • Using original wording for payement from another bank and setting type
  • Check if there is a associated transaction for "Achat de"
  • Fixing bad return
  • Include archived transactions
  • Fix pro accounts transactions amounts not in account currency
  • Fix parsing converted amount from other currency
  • Fixing paypal using different token for conversion
  • Removing commande à from transaction list
  • English key word to skip transactions
  • Fcking paypal

Modules: piratebay

  • Use new domain names

Modules: playme

  • Add a profile walker to automatically send won challenges
  • Implement get_account_status()
  • Fix warnings
  • Answer only to 5 questions
  • Fix challenges with users who have already challenged you
  • Fix crash when there is no contact
  • Use Module.create_browser to correctly give all standard arguments (proxy, logger, etc.)

Modules: poivy

  • Fix low balance listing

Modules: popolemploi

  • Fix search and upgrade to browser2
  • Website uses https
  • Site changed

Modules: presseurop

  • Fix body selectors

Modules: radiofrance

  • Site changed / backend rewritten using browser2
  • Manage selection podcasts / improve testing
  • Handle podcasts
  • Fix indentation typo
  • Minor fixes
  • Cleanly display live in repl
  • Handle new francebleu website, retrieve the good player url when default player is the video player
  • Fix 404 errors in selection

Modules: razibus

  • Increase timeout
  • Support timezone

Modules: redmine

  • Fix crash when parsing custom fields

Modules: regionsjob

  • Fix: site changed
  • Fix parsing when advert is a picture
  • Update to new Join filter
  • Do not crash while parsing an external advert

Modules: sachsen

  • Fix parsing of flow level
  • Improve test for sachsen module
  • Partial fix of new website
  • Fix history with new website

Modules: seloger

  • Remove useless condition
  • Fix bug on pagination
  • Use new DictElement object
  • Handle special characters (#1883)
  • Fix json parsing

Modules: senscritique

  • Skip test when StopIteration is raised
  • Improve module code
  • Fix date parsing
  • Add Mangas missing channel
  • Fix bug when end_date is asked and not found
  • Use set_profile(Firefox()) instead of _setup_session(Firefox())
  • Remove id lists
  • Fix channels selection
  • Site changed (we cannot choose channels packages anymore)

Modules: societegenerale

  • Correctly detect when the website is unavailable
  • Page parsing for saving accounts
  • Update upstream certificate hash
  • Update certificate hashes
  • Credit card is no longer in coming
  • Switch rdate/date and little fixes
  • Update hash of soge enterprise certificate
  • Fix duplicate accounts on cards accounts
  • Adding IBAN support for pro and entreprise accounts
  • Setting new certificat hash
  • Fix greediness of a transaction pattern to avoid crash
  • Update certificate
  • Testing account true id for card
  • Check ssl certificate after browser construction
  • Support market account
  • Support life insurance account (Investment)
  • Refactor
  • Support life insurance account (Transaction)
  • Fix type of 'PEA Espèces' account
  • Detect and log unknown type account
  • Do not iterate over all available transaction
  • Move set_date into LifeInsuranceHistory, and do not return to transactions list
  • Do not store NotAvailable amounts
  • Do not crash on pro account
  • Fix investment valuation
  • Fixing unsuported link account
  • Adding PEA Especes to unsuported link
  • Dd an url to the crashing ones
  • New ssl certificate
  • Add reinitpassword page to prevent crash.
  • Fix parsing and pagination of market investments
  • Try to type more accounts
  • Adding iban number
  • Try to fix weird crash
  • Fix navigation without id transactions
  • Fixing fake id for investment
  • Fixing account type

Modules: tinder

  • Fix warnings
  • Fix crash when a contact has been removed
  • Use APIBrowser
  • Do not crash when the limitation rate is reached
  • fix login on tinder: need to find the cookie in js
  • Avoid aborting profiles walker when queue is empty
  • Fix PEP8 E713
  • use Module.create_browser to correctly give all standard arguments (proxy, logger, etc.)

Modules: transilien

  • Fix _regexp name in RoadMapDuration filter
  • Fix ambiguous departure/arrival in roadmap
  • Use new DictElement object
  • Fix incorrect roadmap duration parsing (#2089)

Modules: twitter

  • Fix: site changed
  • Exclude non-tweets
  • Fix: trendy tweets (site changed)
  • Fix search pagination
  • Fix comments parsing
  • Fix condition to correctly check if user is logged

Modules: vimeo

  • There is no latest category on vimeo (#1692)
  • Allow to browse categories and channels
  • Bump to https and handle site changes
  • isSafeForFamily field is not always filled + correctly handle its value

Modules: vlille

  • Fix encoding error

Modules: wellsfargo

  • Force TLS v1.0 for Wells Fargo module. (#1647)
  • Credit limit, payment due date and amount for Wells Fargo bank (#1720)
  • Scrape minimum credit card payment and due date (#1809)
  • Update to new login security checks (#1999)
  • backport check_output for 2.6

Modules: wordreference

  • [wordreference] Adapt to browser2

Modules: youjizz

  • Min pattern size for search is 3 characters
  • Use UserError instead of ValueError
  • Videos are now hosted on another domain

Modules: youporn

  • 'a' and 'i' pattern are not supported (404 error)
  • Rewrite with Browser 2
  • Use UserError instead of ValueError
  • get_video fill the id

Modules: youtube

  • Fix parsing of js signature (#1695)
  • Backported changes from youtube-dl
  • Fix call to _extract_signature_function()

Weboob 1.0 is out

Posted by Florent Fourcot,

After more than four years of work, we are very proud to announce the first stable release of the Weboob project.

We think that the new browser is ready to use, and stable enough to not change the API during the 1.X branch lifetime. Web scraping is a repetitive task, but we believe that Weboob can now improve that. With all the tools of the Browser (ListElement, Filters, etc) a module can be written in only few lines, we factorize the boring scraping part.

Since the 0.j release, the CapHousing part has been improved. A new option is available for the condition options. We provide a better management of the encoding in applications, in the Weboob class allows now a direct access to the modules, like a python dictionary. There are several new modules, one of them is for Citibank.

If you are using out of tree tools based on Weboob, please upgrade very carefully. We changed many paths in the API, and the first browser has been deprecated.

Credits

General

API Big-Bang

  • Rename BaseBackend to Module
  • Rename BACKEND to MODULE
  • Rename backend.py to module.py
  • Rename BaseApplication to Application
  • Rename CapBase to Capability
  • Rename BasePage to Page
  • Rename BaseBrowser to Browser
  • Move CleanHTML to html filters
  • Remove * imports in filters
  • Move weboob.tools.browser2 to weboob.browser
  • Move weboob.tools.exceptions to weboob.exceptions
  • Move weboob.tools.browser to weboob.deprecated.browser
  • Move weboob.tools.parsers to weboob.deprecated.browser.parsers
  • Move weboob.tools.mech to weboob.deprecated.mech
  • Remove the "backend" result in do() calls

Core

  • Catch the proper exception for missing icon
  • Replace usage of os.mknod() by os.open(O_CREAT)
  • Use the print() function everywhere
  • WebNip.iter_backends takes a new optional parameter 'module'
  • Add getitem on WebNip to get a loaded backend by name
  • Create PrintProgress class instead of using IProgress as default one
  • Allow to load a module with config=None
  • A lot of pep8 fixes

Capabilities

  • Let get_currency guess US$ means USD
  • Prevent mess when copying BaseObject instances

Capabilities: bank

  • Add Investment.description field
  • Add Emirati Dirham AEB currency

Capabilities: calendar

  • Add Conference event category

Capabilities: parcel

  • Add parcelnotfound exception

Capabilities: housing

  • Add and handle in flatboob house_types field
  • Add and handle in leboncoin a new house type: UNKNOWN
  • Adding a url field in housing capability and management of it in flatboob

Applications

  • Add a new debug level (-dd option)
  • Add a " LIMIT " keyword in conditions
  • Centralize encoding guesses, default to UTF-8 (#1352)
  • Use class attributes as much as possible for application output
  • Define std* in the proper class
  • Handle datetime in condition argument
  • os.isatty is now forbidden (as stream.fileno() is not implemented by StringIO)
  • logging: Output to stderr, not stdout
  • logging: better colors

Applications: repl

  • When getting an object, if at least one is found, display errors but correctly return the found object

Applications: boobmsg

  • Fix "show" for threads

Applications: flatboob

  • Ask for query.type in flatboob
  • Add load command
  • Fix bug type_of_good does not exist anymore

Applications: Qflatboob

  • Manage count to avoid problems during pagination

Applications: pastoob

  • Add an option to set a custom file encoding

Applications: parceloob

  • Catch parcelnotfound by untracking

Applicatitons: traveloob

  • Fix: crash if departure time is not available

Applications: videoob

  • Set non verbose mode for wget when downloading m3u8 (fix #1643)

Applications: weboobcfg

  • Return correct exit status code for enable and disable commands

Applications: webcontentedit

  • Better checks for vim usage

Browser

  • Add a way to asynchronously handle requests and pages
  • Backporting mergin_hook to support hook's requests in wheezy
  • HTMLPage checks the inner charset and parse again document if it is not the same than Content-Type HTTP header
  • Add a trivial android profile
  • Add has-class xpath function

Browser: filters

  • Add debug informations
  • Raise ParseError only with None/NotAvailable/NotLoaded values, not with empty strings
  • Add a way to customize sign handling for CleanDecimal
  • Regexp: let template be a callable
  • Add some javascript dedicated filters
  • Add an nth parameter to Regexp filter
  • Add str to _Filters

Browser: elements

  • handle_loaders into AbstractElement
  • Ability to select an ItemElement

DeprecatedBrowser

  • Fix: certificate check on servers which don't allow SSLv3

Documentation

  • Update to the new API
  • Show base classes in documentation

Tools

  • American amount to decimal conversion (ref #1641)
  • PDF decompression function (ref #1641)
  • Regexp-based tokenizer (ref #1641)

Tools: html2text

  • Use the class if possible

Tools: make_man

  • Copyright on top of file

Tools: newsfeed

  • No need for workaround with feedparser>=5.1

Tools: tests

  • Allow changing modules path and adding to PYTHONPATH

Tools: pyflakes

  • Add test to prevent usage of prints in modules
  • Detect deprecated has_key function

Tools: values

  • Ability to set value to an empty string if it is available in choices

Packaging: setup

  • Add futures, avoid Py2-only libs under Py3
  • Use Python3-compatible syntax in debpydep
  • Add ignore dirs for flake8

Contrib: boobot

  • Add a check_twitter method

Contrib: videoobmc

  • Force relative imports

Contrib: weboob-generic (munin script)

  • Add category option

Modules: alloresto

  • Fix: website changes (enable https and fix the form xpath)

Modules: arretsurimages

  • Fix: site changed

Modules: aum

  • Remove useless features of module that don't work anymore
  • Enable https
  • Import exceptions from core

Modules: banqueaccord

  • Support canceled transactions
  • Increase timeout because of slow website

Modules: biplan

  • Use the Python SkipTest if possible

Modules: boursorama

  • Remove prints

Modules: bred

  • Limit length of password
  • Remove lot of old code and keep card transactions in separate card accounts
  • Translating accnum description

Modules: carrefourbanque

  • Do not try to parse useless accounts (closes #1432)
  • Fix: login form is now the second form on the page

Modules: cic

  • Fix: new certificate hash
  • Set an unique id

Modules: cmso

  • Fix: parsing of transaction amounts (strip nbsp)
  • Fix: parsing of huge account balances

Modules: colissimo

  • Fix: return the real error message, not "label"
  • Raise ParcelNotFound in colissimo
  • Return the fullid of not found parcel
  • Upgrade to browser2

Modules: cragr

  • Remove prints
  • Add a regexp for checking password

Modules: creditcooperatif

  • Add unique id to creditcooperatif (perso)
  • Update regexps
  • Use find object
  • Upgrade to browser2 (perso)

Modules: creditmutuel

  • Fix: do not lock browser2 anymore (#1635)

Modules: dresdenwetter

  • Add the debug decorator to dresdenwetter filter

Modules: europarl

  • Remove prints

Modules: feedly

  • Use the Python SkipTest if possible
  • Fix: unicode warning

Modules: fortuneo

  • Do exactly the same thing than js to always get accounts list

Modules: gazelle

  • Fix: infinite loop on fail login, and fix error message lookup

Modules: gdcvault

  • Remove prints

Modules: grooveshark

  • Fix: bug when Year field is empty in grooveshark json
  • Use the Python SkipTest if possible

Modules: hds

  • Convert to browser2 and fix it

Modules: hellobank

  • Remove prints

Modules: hybride

  • Use the Python SkipTest if possible

Modules: imgur

  • Restrict URL to imgur domains

Modules: ing

  • Fix: add an Index for some accounts...
  • Add a test to detect loops in the history
  • Fix: testing of saving accounts
  • Fix: crash on coming operations
  • Add loggedPage on bourse.ingdirect.fr
  • Add a @ckeck_bourse decorator for a clean redirect

Modules: kickass

  • Fix: parsing of torrent titles

Modules: lacentrale

  • Fix: deprecated has_key

Modules: lcl

  • Always raise instances of NotImplementedError

Modules: minutes20

  • Fix: parsing insolite pages

Modules: nettokom

  • Add tests

Modules: okc

  • Remove prints

Modules: oney

  • Add a favicon
  • Add missing symbols for the virtual keyboard
  • Fix: do not crash on months with no transactions

Modules: ouifm

  • Fix: new radio names

Modules: ovs

  • Force relative import

Modules: pap

  • Adapt to browser2
  • Exclude adverts from other websites
  • Fix: image retrieving

Modules: pastebin

  • Fix: crash on spam page

Modules: paypal

  • Use AmericanTransaction.decimal_amount in PayPal module. Part of #1641

Modules: quvi

  • Force relative import

Modules: seloger

  • Adapt to browser2
  • Fix: pagination
  • Fix: obj filling

Modules: societegenerale

  • Remove prints
  • PIL is a global requirement, remove the check

Modules: tinder

  • Fix: auth on tinder by correctly set the User-Agent header

Modules: transilien

  • Fix: crash on late departures

Modules: twitter

  • Fix storage system
  • Fix purge system
  • Do not import Browser1 exception

Modules: unsee

  • Restrict URL to unsee domains

Modules: vlille

  • Better description

Modules: wellsfargo

  • Fix: compatibility with old versions of mechanize
  • Add a favicon
  • Rewrite Wells Fargo with browser2 (closes #1624)
  • Improved Wells Fargo module stability.
  • Use AmericanTransaction.decimal_amount, closest_date, decompress_pdf and ReTokenizer in WellsFargo module. Part of #1641

Modules: youjizz

  • Fix: fillobj on video thumbnail

Modules: youtube

  • Update part of the js interpreter