Merge pull request #23474 from eileenmcnaughton/import_dep

[civicrm-core.git] / release-notes / 5.30.0.md

# CiviCRM 5.30.0

Released October 7, 2020

- **[Synopsis](#synopsis)**
- **[Features](#features)**
- **[Bugs resolved](#bugs)**
- **[Miscellany](#misc)**
- **[Credits](#credits)**
- **[Feedback](#feedback)**

## <a name="synopsis"></a>Synopsis

| *Does this version...?*                                         |         |
|:--------------------------------------------------------------- |:-------:|
| Fix security vulnerabilities?                                   |   no    |
| **Change the database schema?**                                 | **yes** |
| **Alter the API?**                                              | **yes** |
| Require attention to configuration options?                     |   no    |
| **Fix problems installing or upgrading to a previous version?** | **yes** |
| **Introduce features?**                                         | **yes** |
| **Fix bugs?**                                                   | **yes** |

## <a name="features"></a>Features

### Core CiviCRM

- **APIv4 Explorer: Generate short cv code
  ([18089](https://github.com/civicrm/civicrm-core/pull/18089))**

  Improves the APIv4 Explorer user interface by including cv code to run the API
  from CLI.

- **Optimise proximity custom search, by reducing addGeocodingData fn call
  ([18248](https://github.com/civicrm/civicrm-core/pull/18248))**

  Improves performance of the proximity custom search.

- **Support more table names for utf8mb4 conversions and database name
  ([18249](https://github.com/civicrm/civicrm-core/pull/18249))**

  Improves the System.utf8conversion API so that one can specify table patterns
  and databases.

- **distmaker - Autogenerate civicrm-*-patchset.tar.gz
  ([18229](https://github.com/civicrm/civicrm-core/pull/18229))**

  Adds a script to autogenerate `civicrm-*-patchset.tar.gz` instead of having to
  manually generate it every release.

- **Finish allowing use of SSL to connect to database
  (Work Towards [dev/core#1926](https://lab.civicrm.org/dev/core/-/issues/1926):
  [18095](https://github.com/civicrm/civicrm-core/pull/18095))**

  Code cleanup towards the goal of allowing SSL database connections.

- **Profile settings - Add new contacts to a Group? is misleading
  ([dev/core#1950](https://lab.civicrm.org/dev/core/-/issues/1950):
  [18153](https://github.com/civicrm/civicrm-core/pull/18153))**

  Improves user experience by editing the help text and description for the "Add
  new contacts to a Group?" setting.

- **API4: Inconsistent behavior on Role ID
  ([dev/event#37](https://lab.civicrm.org/dev/event/-/issues/37):
  [18285](https://github.com/civicrm/civicrm-core/pull/18285))**

  Allows searching within arrays or serialized strings via APIv4 (and the new
  Search extension).

- **APIv4 - Fix output of CustomValue create/save/update
  ([18195](https://github.com/civicrm/civicrm-core/pull/18195))**

  Improves APIv4 output when saving custom pseudo-entities so that it outputs
  the values saved instead of `[is_error => 0, result => true]`.

- **Add ability to enable query logging per process
  ([18335](https://github.com/civicrm/civicrm-core/pull/18335))**

  Makes it possible to enable logging by process as opposed to all processes or
  no processes.

- **Expose field label to APIv4 and Search creator
  ([18255](https://github.com/civicrm/civicrm-core/pull/18255) and
  [18256](https://github.com/civicrm/civicrm-core/pull/18256))**

  Progress towards codifying the difference between a field's title and
  label in the schema.

  These changes:
   - expose labels to APIv4
   - switch to using labels in the new search extension.
   - separates titles from labels in exportable fields.
   - attempts to clarify the difference between titles and labels in the
   getfields description as:
     - title: "Technical name of field, shown in API and exports"
     - label: "User-facing label, shown on most forms and displays"

- **Metadata fix - phone_type_id, location_type_id, gender_id
  ([18114](https://github.com/civicrm/civicrm-core/pull/18114))**

  Improves metadata for various fields to align with new title and label
  standards.

- **CRM_Utils_VersionCheck - respect force param
  ([18183](https://github.com/civicrm/civicrm-core/pull/18183))**

  Makes the versionCheck job more consistent with other checks by respecting
  APIv4's "force" setting to run even if disabled.

- **Add Serialize key to payment_processor field for Event & Contribution Page
  ([18110](https://github.com/civicrm/civicrm-core/pull/18110))**

  Use metatdata rather than code handling to save array of processors. This
  makes it so that the DAO can handle arrays if it knows the format.

### CiviContribute

- **Shell Financial ACLs extension
  ([18269](https://github.com/civicrm/civicrm-core/pull/18269))**

  Begins work to move Financial ACL code into an extension.

- **Add Line Item v4 API (Work Towards
  [dev/core#1980](https://lab.civicrm.org/dev/core/-/issues/1980):
  [18275](https://github.com/civicrm/civicrm-core/pull/18275) and
  [18293](https://github.com/civicrm/civicrm-core/pull/18293))**

  Extends permission handling for LineItem delete and LineItem ACLs from APIv3 to
  to APIv4 by moving it to the `financialacls` extension.

- **Book keeping Report : Add Time field for Transaction Date and Receive Date
  field ([dev/report#47](https://lab.civicrm.org/dev/report/-/issues/47):
  [18268](https://github.com/civicrm/civicrm-core/pull/18268))**

  Adds a time field to the "Transaction Date" and "Receive Date" filters on the
  Bookkeeping report.

### CiviEvent

- **Wrap "not you" message in crmRegion
  ([18236](https://github.com/civicrm/civicrm-core/pull/18236))**

  Wraps the "Not you, do you want to register for someone else" message in a
  crmRegion so that it can be removed/replaced by the buildForm hook.

### CiviMail

- **Add new columns to mailing summary report
  ([dev/core#1944](https://lab.civicrm.org/dev/core/-/issues/1944):
  [18132](https://github.com/civicrm/civicrm-core/pull/18132))**

  Adds two new columns to the mailing summary report: "Sender Name" and "Sender
  Email".

- **Add disable_smarty option to MessageTemplate.send API
  ([18118](https://github.com/civicrm/civicrm-core/pull/18118))**

  Improves the `MessageTemplate.send` API by adding a new optional parameter
  "disable_smarty", when "disable_smarty" is set to true smarty is not applied
  to the contents of the message.

### WordPress Integration

- **Switch WP over to new installer
  ([dev/wordpress#37](https://lab.civicrm.org/dev/wordpress/-/issues/37):
  [215](https://github.com/civicrm/civicrm-wordpress/pull/215))**

  Switch to new CiviCRM Installer.

### Drupal Integration

- **Allow for the setting of an additional constant CIVICRM_DRUSH_DSN
  ([613](https://github.com/civicrm/civicrm-drupal/pull/613))**

  Allows sites to configure CIVICRM_DRUSH_DSN for drush to use to set up
  database connections.

## <a name="bugs"></a>Bugs resolved

### Core CiviCRM

- **Increase size of mysql field in export table for column with
  CRM_Utils_Type::T_INT type
  ([dev/core#1661](https://lab.civicrm.org/dev/core/-/issues/1661):
  [17956](https://github.com/civicrm/civicrm-core/pull/17956))**

  Fixes some data too long for column errors when exporting.

- **Improve robustness of api/class.api.php
  ([18283](https://github.com/civicrm/civicrm-core/pull/18283))**

  Ensures that the API mode is set to "local" when an empty string is passed
  as the server.

- **add serialize data to actionschedule
  ([18289](https://github.com/civicrm/civicrm-core/pull/18289))**

  Ensures one can create a scheduled reminder via the API that triggers on two
  different events.

- **API - Add upgrade-safe checks to ensure table exists before reading
  ([18135](https://github.com/civicrm/civicrm-core/pull/18135))**

  Adds efficient checks into the API (v3 and v4) to ensure a DAO table
  exists before trying to read from it. This prevents hard-crashes in the UI
  when there are pending updates (or during the upgrade process itself),

- **DB_DataObject - Read CIVICRM_DEBUG_LOG_QUERY correctly
  ([305](https://github.com/civicrm/civicrm-packages/pull/305))**

  Makes The value of `CIVICRM_DEBUG_LOG_QUERY` TRUE-ish or FALSE-ish instead of
  considering any value true.

- **APIv4 Explorer - Fix possible undefined index
  ([18093](https://github.com/civicrm/civicrm-core/pull/18093))**

- **Option value cache key missing domain ID can result in wrong value retrieved
  for domain ([dev/core#1971](https://lab.civicrm.org/dev/core/-/issues/1971):
  [18252](https://github.com/civicrm/civicrm-core/pull/18252))**

- **Upgrade screen - show success instead of error if already upgraded
  ([18181](https://github.com/civicrm/civicrm-core/pull/18181))**

- **trim white space when checking dedupe
  ([18234](https://github.com/civicrm/civicrm-core/pull/18234))**

- **Fix search extension excluded from tarball
  ([18402](https://github.com/civicrm/civicrm-core/pull/18402))**

- **Permissions problem on add or edit template
  ([dev/core#2008](https://lab.civicrm.org/dev/core/-/issues/2008):
  [18437](https://github.com/civicrm/civicrm-core/pull/18437))**

- **Search ext: Aggregate field fixes
  ([18520](https://github.com/civicrm/civicrm-core/pull/18520))**

- **Some datepicker fields only allow choosing dates far in the future
  ([dev/core#2052](https://lab.civicrm.org/dev/core/-/issues/2052):
  [18524](https://github.com/civicrm/civicrm-core/pull/18524))**

### CiviCase

- **Fix js for case type status change on load
  ([18254](https://github.com/civicrm/civicrm-core/pull/18254))**

### CiviContribute

- **Clean money for non-deductible amount
  ([18300](https://github.com/civicrm/civicrm-core/pull/18300))**

  Ensures that the Contribution API cleans data entered into the "non-deductible
  amount" field the same way it cleans the other money fields.

- **Processors - remove gross_amount param from processors
  ([18177](https://github.com/civicrm/civicrm-core/pull/18177))**

  The gross_amount param is returned by a bunch of processors but is not used so
  this removes it.

- **{contribution.receipt_date} token does not use any CiviCRM date formatter,
  output in YYYY-MM-DD HH:MM:SS format and {contribution.receive_date} also uses
  a non-standard format
  ([18176](https://github.com/civicrm/civicrm-core/pull/18176))**

- **Fix PHP notice on contribution page
  ([18116](https://github.com/civicrm/civicrm-core/pull/18116))**

- **E_NOTICE on contribution page widget tab
  ([18189](https://github.com/civicrm/civicrm-core/pull/18189))**

- **change payment instrument when pending payment paid through credit card
  ([dev/core#912](https://lab.civicrm.org/dev/core/-/issues/912):
  [18266](https://github.com/civicrm/civicrm-core/pull/18266))**

- **PayPal Standard redirect fails on WordPress
  ([dev/core#1997](https://lab.civicrm.org/dev/core/-/issues/1997):
  [18525](https://github.com/civicrm/civicrm-core/pull/18525) and
  [18534](https://github.com/civicrm/civicrm-core/pull/18534))**

- **Invoice no longer displays amount paid/due
  ([dev/core#2035](https://lab.civicrm.org/dev/core/-/issues/2035):
  [18602](https://github.com/civicrm/civicrm-core/pull/18602) and
  [18628](https://github.com/civicrm/civicrm-core/pull/18628))**

### CiviEvent

- **Allow the "Cancellation or transfer time limit (hours)" to be negative
  ([dev/event#34](https://lab.civicrm.org/dev/event/-/issues/34):
  [18067](https://github.com/civicrm/civicrm-core/pull/18067))**

  Ensures that entering a negative value into "Cancellation or transfer time
  limit (hours)" saves correctly and works as expected.

- **Batch Update via Profile does not supply data for editing when custom fields
  created after participant added to event
  ([18235](https://github.com/civicrm/civicrm-core/pull/18235))**

- **E_WARNING when saving event fees admin page if there's no discounts set
  ([18169](https://github.com/civicrm/civicrm-core/pull/18169))**

### CiviMail

- **Unexpected behavior from api.MailingEventSubscribe.create
  (Work Towards [dev/mail#24](https://lab.civicrm.org/dev/mail/-/issues/24):
  [18325](https://github.com/civicrm/civicrm-core/pull/18325))**

  Fixes labels for APIv3 `MailingEventSubscribe.create` so that they accurately
  represent the behavior (unsubscribe v subscribe).

- **Mass SMS: Limit group selection to mailing groups
  ([18154](https://github.com/civicrm/civicrm-core/pull/18154))**

  Ensures that the fields "Include Groups" and "Exclude Groups" on the "New SMS" form
  only list groups of the type "Mailing List".

- **Help text bubble is empty and smarty errors when .hlp file is missing for
  settings forms
  ([dev/core#1920](https://lab.civicrm.org/dev/core/-/issues/1920):
  [18648](https://github.com/civicrm/civicrm-core/pull/18648))**

- **Fix error when creating new AB test mailing because domain_id is a required
  field ([18685](https://github.com/civicrm/civicrm-core/pull/18685))**

### CiviMember

- **Fix regression on back-office membership renewal by credit card
  ([18615](https://github.com/civicrm/civicrm-core/pull/18615))**

### Drupal Integration

- **Fix : Enable Drupal Watchdog Logging for Drupal 8
  ([dev/core#1943](https://lab.civicrm.org/dev/core/-/issues/1943):
  [18115](https://github.com/civicrm/civicrm-core/pull/18115))**

  Ensure the setting `Enable Drupal Watchdog Logging` works on Drupal 8 sites.

### WordPress Integration

- **Check if $post defined before trying to use its field values.
  ([18168](https://github.com/civicrm/civicrm-core/pull/18168))**

  This fixes php notices like `PHP Notice:  Trying to get property 'post_name'
  of non-object in <webroot
  path>\wp-content\plugins\civicrm\civicrm\CRM\Utils\System\WordPress.php on
  line 288`

## <a name="misc"></a>Miscellany

- **Fix regen and update civicrm_generated
  ([18273](https://github.com/civicrm/civicrm-core/pull/18273))**

- **Do not pass by reference for createRelatedMemberships
  ([18243](https://github.com/civicrm/civicrm-core/pull/18243))**

- **Typo in groups dropdown on scheduled reminders admin form
  ([dev/core#1956](https://lab.civicrm.org/dev/core/-/issues/1956):
  [18323](https://github.com/civicrm/civicrm-core/pull/18323))**

- **Replace CRM_Utils_Array::value in contribution/confirm and bao/membership
  ([18157](https://github.com/civicrm/civicrm-core/pull/18157))**

- **Cleanup contactID variables in event/contribution register buildQuickForm
  ([18208](https://github.com/civicrm/civicrm-core/pull/18208))**

- **Use already determined value for contributionRecurID
  ([18265](https://github.com/civicrm/civicrm-core/pull/18265))**

- **Minor code cleanup to Order API
  ([18217](https://github.com/civicrm/civicrm-core/pull/18217))**

- **CodeGen - Add use statement to extensions DAO files
  ([18094](https://github.com/civicrm/civicrm-core/pull/18094))**

- **Use more appropriate money format function
  ([18151](https://github.com/civicrm/civicrm-core/pull/18151))**

- **Metadata fix in activity search
  ([18216](https://github.com/civicrm/civicrm-core/pull/18216))**

- **Replace deprecated Drupal 8 constants
  ([18240](https://github.com/civicrm/civicrm-core/pull/18240))**

- **Spelling mistake "separator" not "seperator"
  ([18238](https://github.com/civicrm/civicrm-core/pull/18238))**

- **Remove unused variables from repeatTransaction
  ([18209](https://github.com/civicrm/civicrm-core/pull/18209))**

- **Remove first_contribution key from repeattransaction
  ([18197](https://github.com/civicrm/civicrm-core/pull/18197))**

- **Remove unnecessary transformation of upgrade msg severity
  ([18182](https://github.com/civicrm/civicrm-core/pull/18182))**

- **Remove old dropped items from the schema xml
  ([18244](https://github.com/civicrm/civicrm-core/pull/18244))**

- **Remove 1 of 2 remaining places where relatedObjects is called in
  completeOrder ([18257](https://github.com/civicrm/civicrm-core/pull/18257))**

- **Remove last pass-by-reference in completeOrder signature
  ([18258](https://github.com/civicrm/civicrm-core/pull/18258))**

- **Remove legacy handling for 'fixing' line_item.entity_id
  ([18155](https://github.com/civicrm/civicrm-core/pull/18155))**

- **Remove deprecated code ids['userId']
  ([18156](https://github.com/civicrm/civicrm-core/pull/18156))**

- **Remove 'hack' that overwrites result of searchColumns hook in mailings list
  ([18237](https://github.com/civicrm/civicrm-core/pull/18237))**

- **Remove unused variable
  ([18108](https://github.com/civicrm/civicrm-core/pull/18108))**

- **Remove handling for 4.2 DBs
  ([612](https://github.com/civicrm/civicrm-drupal/pull/612))**

- **Remove ref to logging time
  ([18221](https://github.com/civicrm/civicrm-core/pull/18221))**

- **Remove 2019 deprecated function
  ([18242](https://github.com/civicrm/civicrm-core/pull/18242))**

- **remove dropped tables for activity target and assignment
  ([18280](https://github.com/civicrm/civicrm-core/pull/18280))**

- **Deprecate 'trapException' in DAO::executeQuery
  ([18138](https://github.com/civicrm/civicrm-core/pull/18138))**

- **Deprecate/remove unused checkVersion() functions
  ([18134](https://github.com/civicrm/civicrm-core/pull/18134))**

- **[REF] Ensure that the form param _id is set when adding a new group w…
  ([18250](https://github.com/civicrm/civicrm-core/pull/18250))**

- **[REF] Update recent replacement for CRM_Utils_Array::value
  ([18172](https://github.com/civicrm/civicrm-core/pull/18172) and
  [18453](https://github.com/civicrm/civicrm-core/pull/18453))**

- **[REF] Remove usages of CRM_Utils_Money::format that pass in a blank s…
  ([18142](https://github.com/civicrm/civicrm-core/pull/18142))**

- **[REF] Move isSSLDSN() function to avoid potential problems
  ([18131](https://github.com/civicrm/civicrm-core/pull/18131))**

- **(REF) CRM_Core_Region - Remove unused bits
  ([18139](https://github.com/civicrm/civicrm-core/pull/18139))**

- **[REF] Move Auto DSN Switching into a core function
  ([18270](https://github.com/civicrm/civicrm-core/pull/18270))**

- **[REF] Remove DB Package as it is now deployed using composer
  ([302](https://github.com/civicrm/civicrm-packages/pull/302))**

- **[REF] Deploy Pear DB package using composer
  ([18027](https://github.com/civicrm/civicrm-core/pull/18027) and
  [18473](https://github.com/civicrm/civicrm-core/pull/18473))**

- **[REF] Update subtypeInfo function to leverage getAllContactTypes
  ([17934](https://github.com/civicrm/civicrm-core/pull/17934))**

- **[REF] Fix metadata label for contribution_page_id
  ([18047](https://github.com/civicrm/civicrm-core/pull/18047))**

- **REF Remove unused passbyreference and var from ipn_process_transaction
  ([18311](https://github.com/civicrm/civicrm-core/pull/18311))**

- **[REF] Clean up return variables on updateContributionStatus,
  updatePendingOnlineContribution
  ([18303](https://github.com/civicrm/civicrm-core/pull/18303))**

- **[REF] Move function to delete merged contacts to the Merger class and
  include all instances of the pair
  ([17980](https://github.com/civicrm/civicrm-core/pull/17980))**

- **[REF] Remove most interaction with $objects in completeOrder
  ([18315](https://github.com/civicrm/civicrm-core/pull/18315))**

- **[REF] Fix financial item allocation of negative payments against completed
  payments ([17810](https://github.com/civicrm/civicrm-core/pull/17810))**

- **[REF] Separate and move line-item specific portion of checkTaxAmount to
  LineItem api ([18294](https://github.com/civicrm/civicrm-core/pull/18294))**

- **[REF] [test] Slight increase in test cover & make functions more re-usable
  ([18291](https://github.com/civicrm/civicrm-core/pull/18291))**

- **[REF] simplify retrieval of existing membership on membership renewal form
  ([18296](https://github.com/civicrm/civicrm-core/pull/18296))**

- **[REF] Extract function to get locations to merge, rename 'operation' …
  ([17991](https://github.com/civicrm/civicrm-core/pull/17991))**

- **[REF] Deprecate unused SQL temptable functions
  ([18171](https://github.com/civicrm/civicrm-core/pull/18171))**

- **Minor cleanup in test class
  ([18170](https://github.com/civicrm/civicrm-core/pull/18170))**

- **Add line item acl tests
  ([18274](https://github.com/civicrm/civicrm-core/pull/18274))**

- **Fix test set up to call Order.create to create the correct line items
  ([18279](https://github.com/civicrm/civicrm-core/pull/18279))**

- **Test & remove handling for max_related in renewal form
  ([18295](https://github.com/civicrm/civicrm-core/pull/18295))**

- **Add test to check, remove unnecessary lines
  ([18292](https://github.com/civicrm/civicrm-core/pull/18292))**

- **[Test framework] - Remove test for deprecated way of calling hook invoke
  ([18136](https://github.com/civicrm/civicrm-core/pull/18136))**

- **(NFC) Update various DAO checksums
  ([18184](https://github.com/civicrm/civicrm-core/pull/18184))**

- **[NFC] Add code comments about hidden but still used buttons
  ([18119](https://github.com/civicrm/civicrm-core/pull/18119))**

- **[NFC] [Test] Initial testing on event payment forms.
  ([18112](https://github.com/civicrm/civicrm-core/pull/18112))**

- **[NFC] Update locale over-ride documentation to mention that it is mos…
  ([17919](https://github.com/civicrm/civicrm-core/pull/17919))**

- **(NFC) Add some tests to the `resources` group
  ([18211](https://github.com/civicrm/civicrm-core/pull/18211))**

- **[NFC] Some DAO checksums not updated
  ([18239](https://github.com/civicrm/civicrm-core/pull/18239))**

- **[NFC] Array formatting
  ([18109](https://github.com/civicrm/civicrm-core/pull/18109))**

- **[NFC] IDE cleanup - arrays, single quotes
  ([614](https://github.com/civicrm/civicrm-drupal/pull/614))**

- **[NFC] Update Event DAO Checksum
  ([18361](https://github.com/civicrm/civicrm-core/pull/18361))**

- **[NFC] Fix dates in RelationshipTest
  ([18530](https://github.com/civicrm/civicrm-core/pull/18530))**

- **api_v3_MultilingualTest - Skip Cxn entities
  ([18547](https://github.com/civicrm/civicrm-core/pull/18547))**

- **[NFC] Update Checksum on CustomField DAO
  ([18689](https://github.com/civicrm/civicrm-core/pull/18689))**

## <a name="credits"></a>Credits

This release was developed by the following code authors:

AGH Strategies - Alice Frumin, Andie Hunt; Agileware - Justin Freeman, Pengyi
Zhang; Circle Interactive - Pradeep Nayak; CiviCRM - Coleman Watts, Tim Otten;
CiviDesk - Sunil Pawar, Yashodha Chaku; Dave D; Flinders University of South
Australia - Tom Anderson; JMA Consulting - Monish Deb, Seamus Lee; John
Kingsnorth; Lemniscus - Noah Miller; Lighthouse Consulting and Design - Brian
Shaughnessy; Megaphone Technology Consulting - Jon Goldberg; MillerTech - Chamil
Wijesooriya; MJW Consulting - Matthew Wire; Squiffle Consulting - Aidan
Saunders; Wikimedia Foundation - Eileen McNaughton, Maggie Epps; Wildsight -
Lars Sanders-Green

Most authors also reviewed code for this release; in addition, the following
reviewers contributed their comments:

Artful Robot - Rich Lott; Christian Wach; Circle Interactive - Martin Castle;
Dave D; Fuzion - Jitendra Purohit; Greenpeace Central and Eastern Europe -
Patrick Figel; MJCO - Mikey O'Toole; Semper IT - Karin Gerritsen; Squiffle
Consulting - Aidan Saunders; Tadpole Collective - Kevin Cristiano

## <a name="feedback"></a>Feedback

These release notes are edited by Alice Frumin and Andie Hunt.  If you'd like
to provide feedback on them, please log in to https://chat.civicrm.org/civicrm
and contact `@agh1`.