Always load vars defined in .env files, providing .env.local.php for prod #501

nicolas-grekas · 2018-12-03T12:28:31Z

Q	A
License	MIT

This PR provides a nice experience where .env files are "always" loaded.
That means their purpose is easy to reason about: the committed files define the defaults (.env especially) - and .env.local defines local overrides.

A new .env.local.php is provided for prod, so that performance is not affected.
When .env.local.php is found, other .env files are ignored.

On the deployement side, this means it should be much easier to deal with .env files for people that can't easilly deal with real env vars:

if you use real env vars they always win
logically, .env files are always loaded (to define defaults)
on prod, one can dump a .env.local.php file for max performance

The plan is to add a new flex command to create this file and ease deployments:
composer dump-env prod

ghost

Pull request passes validation.

ghost

Pull request passes validation.

symfony/framework-bundle/3.3/config/bootstrap.php

ghost

Pull request passes validation.

ghost

Pull request passes validation.

nicolas-grekas · 2018-12-10T13:02:35Z

@symfony/deciders votes welcome.

ghost

Pull request passes validation.

ghost

Pull request passes validation.

symfony/framework-bundle/4.2/config/bootstrap.php

ghost

Pull request passes validation.

ghost

Pull request passes validation.

nicolas-grekas · 2018-12-11T14:59:28Z

(updated to use .env.cache.php instead of .env.local.php)

lyrixx · 2018-12-11T15:08:35Z

Things are getting really complicated IMHO. We have so many .env files.
It was much simpler when we were using .yaml file.
I totally understand why we needed to switch to .env file, but IMHO we should rethink everything.

For example why do we need a committed .env file while we say that we need to defined a default value for each env var in the services.yaml ? (or another file). We need a default value for the default value ?
(Actually, this is a rhetorical question. I know the reason: because if we want to use env var (not resolved I mean) even in the DIC we need this .env).

But how many people really need really dynamic values ? very few.
We are making things really complicated because of a few people, leaving so many others behind
There are plenty of questions like "how does env var work", "what should I put in my env var", "how do I change my DB in test" etc etc. Theses question are on the table almost everyday on slack (different slack / different channel).
Most of the people I work with / meet / train or discuss with do not understand where we are going and think it's now more complicated than before.

Symfony 4 + Flex + MakerBundle is really cool and easy. But theses env var really needed to be re-thought

nicolas-grekas · 2018-12-11T15:10:39Z

@lyrixx 💯 - that's what this PR provides :)

ghost

Pull request passes validation.

ghost

Pull request passes validation.

lyrixx · 2018-12-11T16:04:42Z

BTW, this PR is a very good move 👍

About this point (the part "which you should"):

if you use real env vars they always win (which you should - see https://12factor.net/)

I totally disagree. It's like always : it depend. Many applications are hosted on bare metal, and will not move. Using env var make things more complicated because you need to load them for FPM + CLI + CRON + Worker. Using a simple .env file (like you porpose) is much more simple.

So we should not say :

you should use env var.

Declaring parameter in .env for just fine and easy for many!

Simperfit · 2018-12-28T11:07:01Z

👍

symfony/flex/1.0/.env

ghost

Pull request passes validation.

…prod

ghost

Pull request passes validation.

mhujer · 2019-01-04T18:35:30Z

@nicolas-grekas This PR probably causes PHPUnit to fail:

Notice: Undefined index: APP_ENV in C:\dev\htdocs\sf-rec\config\bootstrap.php on line 19

Steps to reproduce:

composer create-project symfony/skeleton sf-rec
cd sf-rec
composer require --dev phpunit/phpunit
vendor\bin\phpunit # or vendor/bin/phpunit on Linux


Notice: Undefined index: APP_ENV in C:\dev\htdocs\sf-rec\config\bootstrap.php on line 19
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.


Time: 93 ms, Memory: 2.00MB

No tests executed!

mhujer · 2019-01-04T18:39:15Z

Nevermind, just found #512

…loaded (weaverryan) This PR was merged into the 4.1 branch. Discussion ---------- Documenting changes recipe 501 - where .env is always loaded See symfony/recipes#501 This basically documents *part* of what is included in #10793. That PR also documents the Flex `composer sync-recipes --force` and `composer dump-env prod` commands, neither of which exist yet. When those are merged / tagged, I'll finish that PR. Note: after merging to 4.2, the URLs at the bottom of `dot-env-changes` need to be changed to point to the 4.2 recipes. Thanks! Commits ------- 203d93a Documenting changes recipe 501 - where .env is always loaded

eschultz-magix · 2019-01-08T07:44:40Z

For example why do we need a committed .env file while we say that we need to defined a default value for each env var in the services.yaml ? (or another file). We need a default value for the default value ?
(Actually, this is a rhetorical question. I know the reason: because if we want to use env var (not resolved I mean) even in the DIC we need this .env).

That's a good point and I also see a duplication in here. From the documentation (https://symfony.com/doc/current/configuration/external_parameters.html):

You can also give the env() parameters a default value: the default value will be used whenever the corresponding environment variable is not found:

# config/services.yaml
parameters:
    env(DATABASE_HOST): localhost

By default, the kernel loads the services.yaml and a services_<env>.yaml for each environment as an extension to the services.yaml. Both files allow to set default values like the .env file. So why is the .env file REALLY necessary? Or on the other hand: Wouldn't it be better to deprecate the env(ENV_NAME) parameter feature then?

Pierstoval · 2019-01-08T09:46:06Z

Default value and Necessary env var are not the same thing. Default value means "If we don't have it, use this". Necessary env var means "We need this env var, and possibly with a valid value".

A .env file is here to specify the necessary values. What it contains "by default" is just values "that should work", but sometimes it does not work.

For instance, check the Recaptcha recipe: Google credentials are mandatory, but they won't break the app if they're not valid. They will break only the portions that actually use a captcha.

…al.php (nicolas-grekas, renanbr) This PR was merged into the 1.2-dev branch. Discussion ---------- Add "dump-env" command to compile .env files to .env.local.php This adds a new command that compiles .env files to .env.local.php: ``` $ composer dump-env prod Successfully dumped .env files in .env.local.php ``` The generated file is used in symfony/recipes#501 Commits ------- 7e561d5 add --empty option 31b12b1 Add "dump-env" command to compile .env files to .env.local.php

Looks like according to these changes: symfony/recipes#501 - Dotenv component is suggested to be added to `require` section of `composer.json`. Both `symfony/skeleton` and `symfony/website-skeleton` require it on prod

This PR was merged into the 3.4 branch. Discussion ---------- Require Dotenv component without --dev Looks like according to these changes: symfony/recipes#501 - Dotenv component is suggested to be added to `require` section of `composer.json`. Both `symfony/skeleton` and `symfony/website-skeleton` require it on prod Commits ------- 3f7594d Require Dotenv component without --dev

…s logic (cristi-contiu) This PR was submitted for the 4.1 branch but it was merged into the 4.4 branch instead. Discussion ---------- [Deployment] Update deployment docs with new .env files logic The [changes from last year regarding how the ``.env`` files are loaded](https://symfony.com/doc/current/configuration/dot-env-changes.html) can affect production deployments. Vendors like [Heroku](https://devcenter.heroku.com/articles/deploying-symfony4#environment-variables) give recommendations based on documentation that no longer applies: "It is recommended that you do not use the symfony/dotenv package in production". The ``symfony/dotenv`` has been moved from ``require-dev`` to ``require``, so the note I removed in this PR no longer applies: symfony/skeleton#122 , symfony/website-skeleton#132 and symfony/demo@7ead1e4 . Loading ``.env`` files in production will work without any changes in composer.json. Putting ``symfony/dotenv`` in ``require-dev`` (or not moving it to ``require`` during an upgrade from older versions of Symfony & recipes) can potentially break your app: if the DotEnv class is missing, the [`config/bootstrap.php` file inside the FrameworkBundle's recipe will throw a RuntimeException](https://github.com/symfony/recipes/blob/master/symfony/framework-bundle/3.3/config/bootstrap.php#L14) . Also see symfony/recipes#501 The note now contains the right way to handle these files in production, by using ``$ composer dump-env prod`` in case you do not want the ``.env`` files to be processed on every request, also suggesting the ``--empty`` flag if you want to rely exclusively on "real" environment variables . Commits ------- b905b57 Update deployment docs with new .env loading

ghost deployed to staging December 3, 2018 12:28 View deployment

ghost approved these changes Dec 3, 2018

View reviewed changes

ghost deployed to staging December 3, 2018 12:31 View deployment

ghost approved these changes Dec 3, 2018

View reviewed changes

nicolas-grekas commented Dec 3, 2018

View reviewed changes

symfony/framework-bundle/3.3/config/bootstrap.php Outdated Show resolved Hide resolved

ghost deployed to staging December 3, 2018 16:14 View deployment

ghost approved these changes Dec 3, 2018

View reviewed changes

ghost deployed to staging December 10, 2018 12:35 View deployment

ghost approved these changes Dec 10, 2018

View reviewed changes

ghost deployed to staging December 10, 2018 13:09 View deployment

ghost approved these changes Dec 10, 2018

View reviewed changes

ghost deployed to staging December 10, 2018 13:18 View deployment

ghost approved these changes Dec 10, 2018

View reviewed changes

This was referenced Dec 11, 2018

[DX] Flaws in the way environment variable are used ? symfony/symfony#27962

Closed

Improve workflow when not using Env vars symfony/flex#287

Closed

weaverryan reviewed Dec 11, 2018

View reviewed changes

symfony/framework-bundle/4.2/config/bootstrap.php Outdated Show resolved Hide resolved

ghost deployed to staging December 11, 2018 14:57 View deployment

ghost approved these changes Dec 11, 2018

View reviewed changes

nicolas-grekas changed the title ~~Always load vars defined in .env files, using .env.local.php for prod cache~~ Always load vars defined in .env files, using .env.cache.php for prod on the web Dec 11, 2018

ghost deployed to staging December 11, 2018 14:58 View deployment

ghost approved these changes Dec 11, 2018

View reviewed changes

ghost deployed to staging December 11, 2018 15:33 View deployment

ghost approved these changes Dec 11, 2018

View reviewed changes

ghost deployed to staging December 11, 2018 15:39 View deployment

ghost approved these changes Dec 11, 2018

View reviewed changes

lyrixx reviewed Dec 28, 2018

View reviewed changes

symfony/flex/1.0/.env Outdated Show resolved Hide resolved

lyrixx approved these changes Dec 28, 2018

View reviewed changes

Simperfit approved these changes Dec 28, 2018

View reviewed changes

maxhelias approved these changes Dec 28, 2018

View reviewed changes

ghost deployed to staging December 28, 2018 18:07 View deployment

ghost approved these changes Dec 28, 2018

View reviewed changes

HeahDude approved these changes Dec 31, 2018

View reviewed changes

Always load vars defined in .env files, providing .env.local.php for …

3e471cb

…prod

ghost deployed to staging January 3, 2019 12:52 View deployment

ghost approved these changes Jan 3, 2019

View reviewed changes

dunglas approved these changes Jan 3, 2019

View reviewed changes

ghost merged commit 3e471cb into symfony:master Jan 3, 2019

ghost pushed a commit that referenced this pull request Jan 3, 2019

Merge pull request #501

f10f2a7

nicolas-grekas deleted the env-always-load branch January 3, 2019 13:09

This was referenced Jan 5, 2019

Documenting changes recipe 501 - where .env is always loaded symfony/symfony-docs#10846

Merged

Allowing APP_ENV to be an empty index #513

Merged

bocharsky-bw mentioned this pull request Jun 8, 2019

Require Dotenv component without --dev symfony/symfony-docs#11703

Merged

guilliamxavier mentioned this pull request Jan 29, 2020

Allow missing APP_ENV in .env.local.php #719

Merged

cristi-contiu mentioned this pull request Feb 8, 2020

[Deployment] Update deployment docs with new .env files logic symfony/symfony-docs#13084

Merged

nanasess mentioned this pull request May 25, 2022

bin/consle でコマンド実行時に.envをロードしないケースがある EC-CUBE/ec-cube#5367

Closed

This pull request was closed.

Uh oh!

Always load vars defined in .env files, providing .env.local.php for prod #501

Always load vars defined in .env files, providing .env.local.php for prod #501

Uh oh!

Conversation

nicolas-grekas commented Dec 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

nicolas-grekas commented Dec 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

nicolas-grekas commented Dec 11, 2018

Uh oh!

lyrixx commented Dec 11, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented Dec 11, 2018

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

lyrixx commented Dec 11, 2018

Uh oh!

Simperfit commented Dec 28, 2018

Uh oh!

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

mhujer commented Jan 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mhujer commented Jan 4, 2019

Uh oh!

eschultz-magix commented Jan 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Pierstoval commented Jan 8, 2019

Uh oh!

Uh oh!

nicolas-grekas commented Dec 3, 2018 •

edited

Loading

nicolas-grekas commented Dec 10, 2018 •

edited

Loading

lyrixx commented Dec 11, 2018 •

edited

Loading

mhujer commented Jan 4, 2019 •

edited

Loading

eschultz-magix commented Jan 8, 2019 •

edited

Loading