1 |
861435e7
|
Andreas Kohlbecker
|
# CDM Data Portal
|
2 |
|
|
|
3 |
|
|
Drupal 7 modules and themes as web frontend to publish data hosted in a [cdm server](https://github.com/cybertaxonomy/cdm-server).
|
4 |
|
|
|
5 |
61710aca
|
Andreas Kohlbecker
|
This project has a monolithic structure, that is it contains multiple projects in one git repository.
|
6 |
861435e7
|
Andreas Kohlbecker
|
|
7 |
61710aca
|
Andreas Kohlbecker
|
Project dependencies required for bundling the final installation package are managed via composer.
|
8 |
|
|
Dependencies of sub-projects like *modules/cdm_dataportal*, *themes/zen_dataportal*, etc still have manually copied dependencies.
|
9 |
861435e7
|
Andreas Kohlbecker
|
|
10 |
|
|
## Project structure
|
11 |
|
|
|
12 |
490421a2
|
Andreas Kohlbecker
|
* `./drush`: commands, configuration and site aliases for Drush
|
13 |
861435e7
|
Andreas Kohlbecker
|
* `./debug`: scripts that help debugging problems with drupal multisite setups
|
14 |
|
|
* `./modules`: drupal modules, the actual **cdm_dataportal** module is found here.
|
15 |
58848045
|
Andreas Kohlbecker
|
* `./scripts/composer`: scripts, installed by composer and manually added ones
|
16 |
|
|
* `./scripts/jenkins-ci`: scripts for testing, release, deployment in jenkins.
|
17 |
861435e7
|
Andreas Kohlbecker
|
* `./site`: documentation source code, will be build by maven (see `pom.xml`)
|
18 |
|
|
* `./src`: java project with selenium tests
|
19 |
|
|
* `./themes`: drupal themes
|
20 |
|
|
|
21 |
|
|
**Generated folders which must not be comitted to the repository:**
|
22 |
|
|
|
23 |
|
|
* `./target`: target folder of maven, used for the side documentation (source in `./site`) and selenium tests (source in `./src`)
|
24 |
|
|
* `./screenshots`: screenshots taken during the selenium test execution
|
25 |
|
|
* `./vendor`: composer cache
|
26 |
|
|
* `./web`: final installation bundle of drupal 7 with modules, themes, profiles, etc. that is created by `composer update`
|
27 |
|
|
|
28 |
|
|
**Creation of the composer project**
|
29 |
|
|
|
30 |
|
|
The parent composer project (`./composer.json`) has been created using the official drupal project template:
|
31 |
|
|
|
32 |
|
|
~~~
|
33 |
|
|
composer create-project drupal-composer/drupal-project:7.x-dev -n dist
|
34 |
|
|
~~~
|
35 |
|
|
|
36 |
|
|
### On using composer with drupal projects
|
37 |
|
|
|
38 |
|
|
https://www.drupal.org/docs/develop/using-composer/using-composer-to-install-drupal-and-manage-dependencies
|
39 |
|
|
|
40 |
|
|
https://www.drupal.org/docs/develop/using-composer/using-composer-with-drupal
|
41 |
|
|
|
42 |
|
|
https://www.drupal.org/docs/develop/using-composer/managing-dependencies-for-a-custom-project
|
43 |
|
|
|
44 |
|
|
|
45 |
547b7def
|
Andreas Kohlbecker
|
|
46 |
5a70408c
|
Andreas Kohlbecker
|
### Installation
|
47 |
|
|
|
48 |
61710aca
|
Andreas Kohlbecker
|
**NOTE:** For detailed instructions which also cover the setup of apache and mysql please refer
|
49 |
|
|
to: https://cybertaxonomy.eu/dataportal/installation
|
50 |
547b7def
|
Andreas Kohlbecker
|
|
51 |
490421a2
|
Andreas Kohlbecker
|
#### Download & extract
|
52 |
|
|
|
53 |
61710aca
|
Andreas Kohlbecker
|
Download the latest release from https://cybertaxonomy.eu/download/dataportal/stable/ to the location where you want
|
54 |
|
|
to install the cdm-dataportal Drupal 7 project e.g.
|
55 |
5a70408c
|
Andreas Kohlbecker
|
|
56 |
|
|
~~~
|
57 |
|
|
cd /var/www
|
58 |
61710aca
|
Andreas Kohlbecker
|
wget https://cybertaxonomy.eu/download/dataportal/stable/drupal-7-cdm-dataportal-5.23.0.tar.gz
|
59 |
5a70408c
|
Andreas Kohlbecker
|
~~~
|
60 |
|
|
|
61 |
61710aca
|
Andreas Kohlbecker
|
This archive contains a shallow clone of the whole project together with a ready to use drupal 7 installation with the
|
62 |
|
|
cdm-dataportal module, zen_dataportal theme and other requirements. The drupal-7 installation is in the sub folder `./web`
|
63 |
5a70408c
|
Andreas Kohlbecker
|
|
64 |
1e12be41
|
Andreas Kohlbecker
|
extract and adapt the ownership of the some folders:
|
65 |
5a70408c
|
Andreas Kohlbecker
|
|
66 |
|
|
~~~
|
67 |
61710aca
|
Andreas Kohlbecker
|
tar -xzf drupal-7-cdm-dataportal-5.23.0.tar.gz
|
68 |
642ac8dd
|
Andreas Kohlbecker
|
./drupal-7-cdm-dataportal/scripts/admin/fix-permissions.sh --web-user www-data
|
69 |
5a70408c
|
Andreas Kohlbecker
|
~~~
|
70 |
|
|
|
71 |
490421a2
|
Andreas Kohlbecker
|
##### Apache2 configuration
|
72 |
|
|
|
73 |
61710aca
|
Andreas Kohlbecker
|
You may now want to copy the apache 2 site configuration files from `scripts/apache2.4/` to `/etc/apache2/sites-available/`
|
74 |
|
|
and to activate one of them, preferably the ssl site configuration:
|
75 |
547b7def
|
Andreas Kohlbecker
|
|
76 |
|
|
~~~
|
77 |
|
|
sudo cp drupal-7-cdm-dataportal/scripts/apache2/dataportal.test* /etc/apache2/sites-available/
|
78 |
c6575e6f
|
Andreas Kohlbecker
|
sudo a2ensite dataportal.test-ssl.conf
|
79 |
|
|
sudo systemctl restart apache2
|
80 |
547b7def
|
Andreas Kohlbecker
|
~~~
|
81 |
|
|
|
82 |
490421a2
|
Andreas Kohlbecker
|
##### Site installation
|
83 |
|
|
|
84 |
547b7def
|
Andreas Kohlbecker
|
Now you are prepared to install a dataportal drupal site.
|
85 |
5a70408c
|
Andreas Kohlbecker
|
|
86 |
547b7def
|
Andreas Kohlbecker
|
A template for the below script can be found in `scripts/user/`
|
87 |
|
|
Make an executable copy from `new-site.sh.template` as `new-site.sh`
|
88 |
5a70408c
|
Andreas Kohlbecker
|
|
89 |
|
|
~~~
|
90 |
547b7def
|
Andreas Kohlbecker
|
cp new-site.sh.template new-site.sh; chmod u+x new-site.sh
|
91 |
5a70408c
|
Andreas Kohlbecker
|
~~~
|
92 |
|
|
|
93 |
547b7def
|
Andreas Kohlbecker
|
Adapt the below shown variables in the script to match your desired setup:
|
94 |
5a70408c
|
Andreas Kohlbecker
|
|
95 |
|
|
~~~
|
96 |
547b7def
|
Andreas Kohlbecker
|
################################################################
|
97 |
|
|
## Configure below variables
|
98 |
|
|
|
99 |
|
|
SITE_NAME='test-site'
|
100 |
|
|
|
101 |
|
|
# HOST_NAME and PROTOCOL determine the base URL of the new site
|
102 |
|
|
# The default values will form the base URL like http://dataportal.test
|
103 |
|
|
# See also MULTI_SITE below
|
104 |
|
|
HOST_NAME='dataportal.test'
|
105 |
|
|
PROTOCOL='https' # values 'http' ot https'
|
106 |
|
|
# For MULTI_SITE=0 the site will be installed at the base BASE_URL
|
107 |
|
|
# In multisite setups (MULTI_SITE=1), however, the site URL results
|
108 |
|
|
# in http://dataportal.test/test-site
|
109 |
|
|
# !! Mutisite support ist still experimental !!
|
110 |
|
|
MULTI_SITE=0 # values: 1 = true, 0 or other = false
|
111 |
5a70408c
|
Andreas Kohlbecker
|
|
112 |
547b7def
|
Andreas Kohlbecker
|
ADMIN_USR='admin'
|
113 |
|
|
ADMIN_PWD='change--me'
|
114 |
|
|
ADMIN_EMAIL='admin@dataportal.test'
|
115 |
5a70408c
|
Andreas Kohlbecker
|
|
116 |
547b7def
|
Andreas Kohlbecker
|
MYSQL_USR='root'
|
117 |
|
|
MYSQL_PWD='change--me'
|
118 |
|
|
DB_PREFIX='drupal7_dataportal_'
|
119 |
5a70408c
|
Andreas Kohlbecker
|
|
120 |
|
|
|
121 |
547b7def
|
Andreas Kohlbecker
|
################################################################
|
122 |
|
|
~~~
|
123 |
|
|
|
124 |
ac9d4d38
|
Andreas Kohlbecker
|
Execute the script **from within the folder** `scripts/user/`, otherwise the `$DRUPAL_ROOT` variable will not match the `./web` folder !
|
125 |
5a70408c
|
Andreas Kohlbecker
|
|
126 |
547b7def
|
Andreas Kohlbecker
|
Once the script has fished it will print out the final URL of the new site together with other useful information.
|
127 |
5a70408c
|
Andreas Kohlbecker
|
|
128 |
490421a2
|
Andreas Kohlbecker
|
### Update - method 1
|
129 |
|
|
|
130 |
61710aca
|
Andreas Kohlbecker
|
**Strategy**: Downloading of the installation package for a new release of the cdm-dataportal and replacing the old
|
131 |
|
|
installation by the content of the installation package.
|
132 |
490421a2
|
Andreas Kohlbecker
|
|
133 |
|
|
* Pro: Few simple steps.
|
134 |
49f7cc43
|
Andreas Kohlbecker
|
* Con: The steps described here are not suitable for multi-site installation or when additional modules are installed.
|
135 |
490421a2
|
Andreas Kohlbecker
|
|
136 |
|
|
~~~
|
137 |
|
|
cd /var/www
|
138 |
|
|
mv drupal-7-cdm-dataportal drupal-7-cdm-dataportal.last
|
139 |
|
|
~~~
|
140 |
|
|
|
141 |
|
|
remove old installation packages
|
142 |
|
|
|
143 |
|
|
~~~
|
144 |
e4a638b8
|
Andreas Kohlbecker
|
rm drupal-7-cdm-dataportal*.tar.gz*
|
145 |
490421a2
|
Andreas Kohlbecker
|
~~~
|
146 |
|
|
|
147 |
|
|
now follow the steps in the chapter **Download & extract** above.
|
148 |
|
|
|
149 |
|
|
finally copy the default site to the new installation
|
150 |
|
|
|
151 |
|
|
~~~
|
152 |
|
|
cp -r drupal-7-cdm-dataportal.last/web/sites/default/ drupal-7-cdm-dataportal/web/sites/
|
153 |
|
|
~~~
|
154 |
|
|
|
155 |
2776ab75
|
Andreas Kohlbecker
|
apply any pending database updates
|
156 |
|
|
|
157 |
|
|
~~~
|
158 |
|
|
drush updatedb
|
159 |
|
|
drush cc all
|
160 |
|
|
~~~
|
161 |
|
|
|
162 |
49f7cc43
|
Andreas Kohlbecker
|
Once you have confirmed that the updated installation is working correctly:
|
163 |
e4a638b8
|
Andreas Kohlbecker
|
|
164 |
|
|
~~~
|
165 |
|
|
rm -r drupal-7-cdm-dataportal.last
|
166 |
|
|
~~~
|
167 |
|
|
|
168 |
490421a2
|
Andreas Kohlbecker
|
### Update - method 2
|
169 |
|
|
|
170 |
49f7cc43
|
Andreas Kohlbecker
|
**Strategy**: In-place updating of the installation by making use of git and drush or composer.
|
171 |
490421a2
|
Andreas Kohlbecker
|
|
172 |
e34c07ee
|
Andreas Kohlbecker
|
*THIS METHOD IS RECOMMENDED FOR MOST SITUATIONS**
|
173 |
|
|
|
174 |
49f7cc43
|
Andreas Kohlbecker
|
* Pro: Update of drupal and custom modules independent of what is provided by the installation package.
|
175 |
e34c07ee
|
Andreas Kohlbecker
|
* Con: Initial preparation is more complex,
|
176 |
ac9d4d38
|
Andreas Kohlbecker
|
|
177 |
|
|
#### Preparation
|
178 |
|
|
|
179 |
0dfee367
|
Andreas Kohlbecker
|
**Install composer v 1.10.x**
|
180 |
ac9d4d38
|
Andreas Kohlbecker
|
|
181 |
0dfee367
|
Andreas Kohlbecker
|
!!! *Composer v2.x would fail to preserve existing site installations* !!!
|
182 |
490421a2
|
Andreas Kohlbecker
|
|
183 |
61710aca
|
Andreas Kohlbecker
|
In case the version provided by apt is too old (<1.10), you can install composer manually in the project
|
184 |
|
|
directory `/var/www/drupal-7-cdm-dataportal`. In the following we assume you have installed composer this way and
|
185 |
|
|
will use `./composer` instead of `composer`.
|
186 |
490421a2
|
Andreas Kohlbecker
|
|
187 |
ac9d4d38
|
Andreas Kohlbecker
|
~~~
|
188 |
0dfee367
|
Andreas Kohlbecker
|
cd /var/www/drupal-7-cdm-dataportal
|
189 |
|
|
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
|
190 |
|
|
php -r "if (hash_file('sha384', 'composer-setup.php') === '756890a4488ce9024fc62c56153228907f1545c228516cbf63f885e036d37e9a59d27d63f46af1d4d07ee0f76181c7d3') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
|
191 |
|
|
php composer-setup.php --filename=composer --version 1.10.22
|
192 |
|
|
php -r "unlink('composer-setup.php');"
|
193 |
ac9d4d38
|
Andreas Kohlbecker
|
~~~
|
194 |
|
|
|
195 |
49f7cc43
|
Andreas Kohlbecker
|
The above commands fails with "Installer corrupt"? Please update the corresponding command with the new one from [http//getcomposer.org/download/](http//getcomposer.org/download/). It is **important not use use a version > 1.*!**
|
196 |
490421a2
|
Andreas Kohlbecker
|
|
197 |
49f7cc43
|
Andreas Kohlbecker
|
**Un-shallow the installation package**
|
198 |
ac9d4d38
|
Andreas Kohlbecker
|
|
199 |
61710aca
|
Andreas Kohlbecker
|
The installation package contains a shallow clone of the git repository only. In order to easily upgrade the
|
200 |
|
|
dataportal-module it is highly recommended to un-shallow the git repository clone
|
201 |
ac9d4d38
|
Andreas Kohlbecker
|
|
202 |
|
|
~~~
|
203 |
|
|
git fetch --unshallow
|
204 |
|
|
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
|
205 |
|
|
git fetch origin
|
206 |
|
|
git checkout origin/master
|
207 |
|
|
~~~
|
208 |
|
|
|
209 |
e34c07ee
|
Andreas Kohlbecker
|
**install dependencies**
|
210 |
|
|
|
211 |
|
|
This will also install **drush**
|
212 |
|
|
|
213 |
|
|
~~~
|
214 |
|
|
./composer install --no-dev
|
215 |
|
|
~~~
|
216 |
|
|
|
217 |
ac9d4d38
|
Andreas Kohlbecker
|
#### Updating to a specific cdm-dataportal release
|
218 |
|
|
|
219 |
61710aca
|
Andreas Kohlbecker
|
NOTE: The modules and themes in the `./web/sites/all/` are sym-linked to the source code in `./modules` and `./themes`.
|
220 |
|
|
Hence, updating the dataportal only involves checking out the according release from the git remote:
|
221 |
ac9d4d38
|
Andreas Kohlbecker
|
|
222 |
|
|
To see the list of available release tags:
|
223 |
|
|
|
224 |
|
|
~~~
|
225 |
|
|
git tag --list | egrep '[0-9]+\.[0-9]+\.[0-9]+' | sort -V
|
226 |
|
|
~~~
|
227 |
|
|
|
228 |
61710aca
|
Andreas Kohlbecker
|
Checkout the release tag. For example to checkout the release `5.23.0`:
|
229 |
ac9d4d38
|
Andreas Kohlbecker
|
|
230 |
|
|
~~~
|
231 |
61710aca
|
Andreas Kohlbecker
|
git checkout 5.23.0
|
232 |
1e12be41
|
Andreas Kohlbecker
|
scripts/admin/fix-permissions --web-user www-data
|
233 |
ac9d4d38
|
Andreas Kohlbecker
|
~~~
|
234 |
|
|
|
235 |
49f7cc43
|
Andreas Kohlbecker
|
Git will respond with a warning that "*You are in 'detached HEAD' state.*", this is OK and no need to be concerned.
|
236 |
ac9d4d38
|
Andreas Kohlbecker
|
|
237 |
490421a2
|
Andreas Kohlbecker
|
Finally apply any pending database updates and clear the cache. **NOTE**: Below we assume that the command is being
|
238 |
61710aca
|
Andreas Kohlbecker
|
executed in the installation package root e.g. `/var/www/drupal-7-cdm-dataportal` and that the site is available under
|
239 |
|
|
https://dataportal.test. Please adapt to your specific settings if needed.
|
240 |
490421a2
|
Andreas Kohlbecker
|
|
241 |
|
|
~~~
|
242 |
e34c07ee
|
Andreas Kohlbecker
|
./vendor/drush/drush/drush -r /var/www/drupal-7-cdm-dataportal/web/ -l https://dataportal.test updatedb
|
243 |
|
|
./vendor/drush/drush/drush -r /var/www/drupal-7-cdm-dataportal/web/ -l https://dataportal.test cc all
|
244 |
490421a2
|
Andreas Kohlbecker
|
~~~
|
245 |
|
|
|
246 |
e34c07ee
|
Andreas Kohlbecker
|
#### Updating drupal and modules
|
247 |
|
|
|
248 |
32a930c7
|
Andreas Kohlbecker
|
The drupal-7-cdm-dataportal installer provides a script for convenient and secure updating of single or multisite setups.
|
249 |
|
|
It will
|
250 |
|
|
|
251 |
|
|
1. create backups
|
252 |
|
|
1. set the site(s) to maintenance mode
|
253 |
|
|
1. Update drupal core and contributed modules via `composer`
|
254 |
|
|
1. Run any pending database updates and clear the caches
|
255 |
|
|
|
256 |
|
|
A brief help for this script is available from `scripts/admin/update-dependencies.sh --help`
|
257 |
e34c07ee
|
Andreas Kohlbecker
|
|
258 |
|
|
~~~
|
259 |
32a930c7
|
Andreas Kohlbecker
|
USAGE: update-dependencies.sh [--deactivate-install] [--multi-site] [--mailto <ADDRESS>]
|
260 |
|
|
--deactivate-install : The install.php will be hidden by appending '.off' to the filename
|
261 |
|
|
-h, --help: Print this help text
|
262 |
|
|
--mailto <ADDRESS>: send a email to the ADDRESS with a log of the update process
|
263 |
|
|
--multi-site: Do a multi-site update. Requires dataportals-drush.
|
264 |
|
|
See https://dev.e-taxonomy.eu/svn/trunk/server-scripts/dataportal-admin/
|
265 |
|
|
--site-url: The site url to be used with drush. This option disables the --multi-site option
|
266 |
e34c07ee
|
Andreas Kohlbecker
|
~~~
|
267 |
|
|
|
268 |
32a930c7
|
Andreas Kohlbecker
|
|
269 |
|
|
|
270 |
|
|
Update a single site installation with default site URL
|
271 |
e34c07ee
|
Andreas Kohlbecker
|
|
272 |
|
|
~~~
|
273 |
32a930c7
|
Andreas Kohlbecker
|
scripts/admin/update-dependencies.sh --deactivate-install
|
274 |
e34c07ee
|
Andreas Kohlbecker
|
~~~
|
275 |
|
|
|
276 |
32a930c7
|
Andreas Kohlbecker
|
Update a single site installation with custom site URL
|
277 |
e34c07ee
|
Andreas Kohlbecker
|
|
278 |
|
|
~~~
|
279 |
32a930c7
|
Andreas Kohlbecker
|
scripts/admin/update-dependencies.sh --deactivate-install --site-url http://edit.test/d7/cichorieae/
|
280 |
|
|
~~~
|
281 |
|
|
|
282 |
|
|
Update a multi-site installation with custom site URL
|
283 |
|
|
|
284 |
|
|
~~~
|
285 |
|
|
scripts/admin/update-dependencies.sh --deactivate-install --multi-site
|
286 |
2776ab75
|
Andreas Kohlbecker
|
~~~
|