Revision as of 15:07, 6 April 2016

Security

Login session variables

REMEMBER that these variables also needs to be set (or at least managed) when emulating and unemulating.

Other session variables generally used

Only lasting during the page execution for all pages that goes through the security check.

Other global variables generally used

Note that the Centralizer system uses $GLOBALS['runtime']['cache']

Access levels

Each usergroup must also have a numeric value because of the login system's way of behaviour.

Access levels from 70-79 and 0-10 should on the pages be set specifically (in $requiredUserLevel) to avoid unwanted inherited permissions.

• *) mostly same as administrators but only within selected date blocks, and no access to certain system wide features
• **) planned access levels - but probably just make it an extension to their main_people record - to avoid multiple l

When creating new access levels use this checklist:

• try to make unique abbreviations/access level names so they can be searched for
• add to table list_accesslevels
• add to get_user_main_menu(), format_accesslevel(), get_common_table_info()
• add to js_functions_phpmaker.js (syncLocalToAccesslevel() function)
• add to main_ext_loginslist.php (in section "Show mappings/localID")
• add to temp_userslist.php (add to switch() making the $emulate_qstr variable)
• add to php_functions_login.php => format_accesslevel()
• add permission and setup in set_pw.php to emulate and set user/password (if emulation should be possible)

Table permissions

Important notes: No records must be deleted if it has attached records in underlying levels Divisions From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Date blocks From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Unions From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Conferences/missions From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Orientations sites From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Note: Only admin can edit orientation date and city Hotel cities From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Campaign sites From divisions and down these permissions are only allowable within their dateblock, and only themselves on the same level

Field permissions

- only admin is allowed to change Active-fields - password field may never be entered/changed manually

File and folder permissions

Modify (666): php/files/*.* except index.php

Modify (777): php/files/logs_scheduled_jobs

Modify (777): php/files/satellite_exports

Modify (777): php/files/temp

Modify (777): php/files/uploads_shortlife

Modify (777): php/files/var_dumps

Modify (777): php/docs

Modify (777): php/docs -ALL SUBFOLDERS-

Modify (666): php/docs/exec_directors_B84H81BN8KBS8F/*Planning_Form*.xls

Modify (777): php/multimedia/ -ALL SUBFOLDERS -

Modify (666): php/multimedia/ -ALL FILES except index.php -

Modify (777): php/pdf_generated/ -ALL SUBFOLDERS-

Modify (666): php/pdf_generated/ -ALL FILES IN SUBFOLDERS- (in order to be able to overwrite them)

Modify (666): php/supportchat/log.txt

Modify (666): php/includes/jensenfw/*.js|*.js.php -ALL FILES - (for phpJSO to obfuscate the files)

Modify (666): php/includes/*.js|*.js.php -ALL FILES except those skipped in _obfuscate_js.php - (for phpJSO to obfuscate the files)

Modify (666): php/error_log (PHP's error logging (warnings, parse errors etc))

Modify ??? : OpenID filestorage location

Scheduler

• checking if we have all necessary information in time before campaigns begin
• sending reminders to people

Design concept

General guidelines

• if information is missing write something like "Awaiting information..."
• hide all e-mail addresses for public users and also for logged in users where they don't need to know the address

Date block branding images

• Trip detail page banner: 960 x 389 (larger is okay as long as ratio is kept)
• Trip overview page thumbnail (https://sharehim.org/upcoming-trips/): 290 x 170 (exact required)
• Email header banner: 564 x 168 (exact required)

They are uploaded to the WordPress Media Library as any other image there.

Rules, standards, notes, etc.

Abbreviations & definitions (terminology/glossary)

Applicant categories

A description of the different categories of applicants (IMPORTANT: Remember to consider co-speaker option, too).

Notation in brackets means is for a co-speaker in that category. Only used where applicable and where there is a difference.

Common for all:

• a record in main_personevents

Operations allowed at the different stages in the application process

Possible combinations of 'category' and 'cospeaker' and their allowed selection of dateblocks

This analyses is done to make sure that we have encompassed all possible combinations and their differences and validity.

Some general definitions of which the following analyses is based on:

• we have 3 different definitions of allowed dateblock selections:
  • dateblocks within a college
  • dateblocks that still has open sites (see definition above under 'Abbreviation & definitions')
  • dateblocks that has not passed their closing date, and therefore still accepts applications
• a cospeaker from a college, can only select date blocks defined for that same college (similar to a def. in next analyses)

Complete definition table

Definitions encompassing all derived conclusions from the above table

Possible combinations of speaker's support team members and their dateblock link (for when selecting members)

This analyses is done to make sure that we have encompassed all possible combinations and their differences and validity.

Some general definitions of which the following analyses is based on:

• a cospeaker from a college, can only be a cospeaker to a speaker from that same college (similar to a def. in former analyses)
• a speaker can have 2 types of support team members:
  • stm (normal)
  • pastor|layman|academy (but not college) cospeakers
• of course a non-cospeaker cannot be a support team member
• of course stm & organizers cannot be cospeakers - the dateblock link of college students is, contrary to the other types, stored in collegedateblocks

Complete definition table

Definitions encompassing all derived conclusions from the above table Note: keeping the dateblockID link in collegedateblocks makes it possible to move all college students from one college to another dateblock with a simple change (considering strict normalization of databases this is the correct way to do it). On the other hand it complicates when dealing with dateblockID for speakers. (At the point of time of writing this the question is whether to do one or the other - whether I already have figured out the ways to go about this difference - which I think I have... since I have made SQL queries for both finding all speakers within a dateblock and finding all stm within a dateblock)

Update: To simplify the SQL statement the field eff_dateblockID (effective dateblockID)

Organizer's responsibility group/scope

NOTE: This is our goal but it doesn't mean that the system is working exactly like this yet! Some development probably needed in this area. An organizer can be responsible for the following groups of people depending on his setup:

For his group of people the areas he can work in include (but is not limited to):

- maintaining their personal record
- entering flight information
- making hotel room reservation
- booking or unbooking graphics equipment
- probably also assigning/unassigning team members???

Multiple events

- on the people's own menu they work with one event at a time - they select themselves which one they want to work with

Website Layout & Coding Format

• generally aim at a minimum screen resolution of 1024x768 pixels
• most website users use IE so it's must be compatible with the newest versions
  • but most browsers can be expected (because of a large user group)
• a menu area and a main area
• light background color/texture
• use default font sizes/families and styles unless something needs to be emphasized or de-emphasized
  • use < b >
  • use or the CSS class "dimmed" for less important text
• don't follow coding structure of main_*.php and list_*.php (generated by PHPMaker) but rather story_menu_admin.php
  • complexity of search and sorting features needs to be considered for every case (sometimes needed, otherwise just simple)
• standard date format: mm/dd/yyyy
  • started writing LOCAL-DATE places where we eventually will localize the date format for user's country
    • consider if strftime() (or strftime_new() in adventsangerne.no) can be used for something
• standard number format: ##,###,###.##
• always use require_function() for defining functions to use. Never include file directly.
• when validating date use $err_occurs_on_page to count how many errors occur. The validation functions also uses this variable.
• recommended to use query string variable "act" when requesting a page with a command to do some operation
• see additional coding format specifications in "Programming principles - Allan.txt"

E-mails

• e-mail addresses must never be shown to the public. They must be presented with a form to be filled out instead.
• general e-mails should bounce to bounce@sharehim.org, while all other e-mails should bounce to the sender address
• format of the unique Mailer ID that can be used in e-mails: X-Mailer-ID: #SRCID#MAIL#FEST07-2#PID686#

Check queries

• IMPORTANT: check that all speakers in a college within a cluster have personevents.clusterID filled out accordingly
• check that all homeland date blocks have at least one conference defined in main_dateblockconfs
  • SELECT main_dateblocks.* FROM main_dateblocks LEFT JOIN main_dateblockconfs USING (dateblockID) WHERE homeland = 1 AND main_dateblockconfs.conferenceID IS NULL ORDER BY title
• conferences, unions etc. have their upper level record
• orphaned group members
• speakers both being group leader and group member
• check that group date block matches the members date blocks
• speaker who haven't been issued any material (use code from _db-query.php)
• check that STMs that are member of teams are also member of that date block
• check that no delete-marked events are found in main_people.active_personeventID
• delete active_personeventIDs that do not exist in main_personevents (maybe obsolete now)
  • _db-query.php: "CLEAR ACTIVE_PERSONEVENTIDs THAT ARE NOT EXISTING IN PERSONEVENTS"
• check that all people in homeland campaigns are members of clusters
• check date block dates:

SELECT dateblockID, title, opening_date, appl_closed_date, data_lock_date, TO_DAYS(closing_date) - TO_DAYS(opening_date) AS days_duration, TO_DAYS(opening_date) - TO_DAYS(appl_closed_date) AS days_before_appl_close, TO_DAYS(closing_date) - TO_DAYS(data_lock_date) AS days_lock_date_before_closingdate
FROM main_dateblocks
WHERE
    TO_DAYS(closing_date) - TO_DAYS(opening_date) <> 15 /* days_duration */
    OR TO_DAYS(opening_date) - TO_DAYS(appl_closed_date) < 0 /* days_before_appl_close */
    OR TO_DAYS(opening_date) - TO_DAYS(appl_closed_date) > 90 /* days_before_appl_close */
    OR TO_DAYS(closing_date) - TO_DAYS(data_lock_date) < -10 /* days_lock_date_before_closingdate */
ORDER BY opening_date

• check pe_eff_siteID integrity:

SELECT main_personevents.personeventID, main_personevents.personID, main_sites.siteID, pe_eff_siteID, main_personevents.categorymoved, main_personevents.cospeaker, main_personevents.eff_dateblockID
FROM main_personevents
LEFT JOIN main_sites ON main_personevents.personeventID = main_sites.personeventID 
WHERE (main_personevents.pe_eff_siteID <> main_sites.siteID
OR (main_personevents.pe_eff_siteID IS NULL AND main_sites.siteID IS NOT NULL) 
OR (main_personevents.pe_eff_siteID IS NOT NULL AND main_sites.siteID IS NULL))
AND cospeaker = 0 AND categorymoved NOT IN ('stm', 'organizer')

• check if we have any visits with any of the user agents currently listed in class webstory -> $skip_useragent_keywords
• occasionally removed unused queries:
  • delete_query(false, array('queryID' => 0, 'queryID_compare' => '>=')); //it will auto probibit deleting used queries
• check that pastor e-mail addresses match e-mail address in our records (system should prohibit it though so any records shown here might be a result of a problem in the system. Also see e-mails with subject "E-mail changed" of 2010-09-06 and 2011-03-10):
  • SELECT main_people.personID, legal_firstname, legal_lastname, main_people.email, main_officers.email AS conf_email, pw FROM main_people INNER JOIN main_officers ON main_people.personID = main_officers.personID WHERE main_people.email <> main_officers.email

Dropdowns

• dropdowns binding a record to it's upper level must only show the upper level values within that branch of the tree

Boolean fields

All boolean fields (yes/no) have the field type TINYINT.

• 1 = yes / true
• 0 = no / false

This makes it easier to determine false and true in PHP.

Some fields also have additional special values like -1 or 2.

Use the label "- not set -" if the boolean value has not been set.

Adding/removing fields for, or links to, person record in main_people

• when deleting a person (main_persondelete.php and delete_person() ) consider this new field
• when merging a person (merge_person() ) consider this new field
• for foreign keys linking to main_people.personID, add a reference in person_alldata.php
• consider necessary changes to ShareHim Satellite

Deleting applicants

Deleting an applicant (meaning that he doesn't want to participate in the program anyway or he has been denied) does not result in an actual removal of the person from the database. We only delete the personevent record.

Deactivating records

To deactivate a record would be a very rare occurrence, only for example if a division re-organizes it's unions. In that case we deactive the unions that are no longer existing and by that we also cancel all current activities for those unions, like equipment booking and scheduled material shipments. This must/can then be re-booked and re-scheduled for the new unions created in stead. This is an example of the principle of deactivating records and theirs effects on the system.

User menus

• Procedures (= links) that aren't applicable at present should be grayed out
  • e.g.: only make active link to match speakers if there are any speakers that need to be matched
  • This will make an easy todo-list overview for the user
  • Group these kind of procedures together on the menu

External links and webservice consumers

The following external places link into pages on the website:

The following systems are calling webservices on our website:

Address formats

3 different formats (currently not differencing USA/Canada):

• USA
  • Street/PO
  • City, State, Zip
  • Country
• Canada
  • Street/PO
  • City, Province, X#X-#X#
  • Country
• Other
  • Street/PO
  • Postal information
  • Country

Telephone formats

On forms where people enter their phone number they need to enter the number WITHOUT country code and WITH area number.

On forms where people edit their phone number we will show a plus (+) and the country code in front of that number. We can do that since we have the country code for all countries in the table list_countries.

Log actions

Used in `sharehim_log` database in table `system_operations`, fields `action` and `subaction`

Table might not yet be complete. (use SQL from file "Code pieces/SQL statements.txt" to make it complete)

IMPORTANT NOTE:

• to minimize number of records the following log actions are deleted once in a while (after taking a local backup of the whole database so we have the information just in case we need it):

DELETE FROM system_operations WHERE `action` = 'campaign site' AND subaction = 'added'; /* date added is registered in main_sites anyway */

DELETE FROM system_operations WHERE `action` = 'login' AND subaction = 'success';

DELETE FROM system_operations WHERE `action` = 'login' AND subaction = 'failed';

DELETE FROM system_operations WHERE `action` = 'login' AND subaction = 'emulated';

DELETE FROM system_operations WHERE `action` = 'logout' AND subaction IS NULL;

Change Log prefixes

The following prefixes are used when entering comments in the CVS repository (see notes below the table):

ADD: When features/functional behaviours are added

CHG: When features/functional behaviours are changed

FIX: When a bug has been fixed

IMPROV: When features/functional behaviours have been improved (and it's neither a decided FIX, CHG, or ADD)

REMOVE: When a functional behaviour or out-dated code has been removed

UNDO: Undo a previous commit (usually the previous one of that file)

Any prefix can have " NWM" appended when it is a change that is "Not Worth Mentioning".

Any prefix can have " MAJOR" appended when it involves a major functionality change or upgrade.

The prefix is following by a colon, a space, and then the description. Examples:

ADD: link to change password

FIX NWM: corrected spelling

ADD MAJOR: donation management system

In very rare cases a prefix is not applicable and therefore not used.

If only functions have been changed we write a comma-seperated list of their names with "()" appended. For example:

ADD: get_dateblocks(), get_dateblock_info(): also return mediafolder field

If both functions and code in other places have been changed, we write the description normaly, but in the end list the functions that have been changed and/or added. For example:

ADD: interface for the new date block field mediafolder. Functions modified: get_dateblocks(), set_dateblock(). Functions added: get_mediafolder()

The same applies for JavaScript functions, but then we write "JS" with the parenthesis, for example: get_field_value(JS)

For PHP classes we write like this: 'class webservice_server'

The point here is to always add the names of functions and classes that are changed, so that it's possible to search CVS for all changes regarding those functions/classes.

Possible improvement (=not yet used): there should actually also be a keyword for deciding whether or not the entry should be seen for the general users of the website, so they know which changes have been made to the website that affect their work. Eg. "technical modification" or "functional modification".

Code-related tags

Codes that are put different places in the source code to link together different pieces of code that are related - to make it easier to maintain it and remember other places where we are dealing with the same thing. It's important that these tags are unique through ALL text files in the project so that they are easy to search for. Always write them in upper case.

System entry points

Places where user will enter this system (might not be a complete list - check also with login.php)

• index.php
• login.php
• menu*.php
• recommend.php (recommenders go directly to here)

SQL sentences

Relationships up through the main line:

main_translations

INNER JOIN main_sites ON main_translations.siteID = main_sites.sideID

INNER JOIN main_hotels ON main_sites.hotelID = main_hotels.hotelID

INNER JOIN main_conferences ON main_hotels.conferenceID = main_conferences.conferenceID

INNER JOIN main_unions ON main_conferences.unionID = main_unions.unionID

INNER JOIN main_dateblocks ON main_unions.dateblockID = main_dateblocks.dateblockID

INNER JOIN main_divisions ON main_dateblocks.divisionID = main_divisions.divisionID

Return all sites for a orientation/dateblock (the opposite direction of above):

SELECT main_sites.*

FROM main_orientations

INNER JOIN main_dateblocks ON main_orientations.dateblockID = main_dateblocks.dateblockID

INNER JOIN main_unions ON main_dateblocks.dateblockID = main_unions.dateblockID

INNER JOIN main_conferences ON main_unions.unionID = main_conferences.unionID

INNER JOIN main_hotels ON main_conferences.conferenceID = main_hotels.conferenceID

INNER JOIN main_sites ON main_hotels.hotelID = main_sites.hotelID

UNION query to obtain all users:

SELECT coord_email AS email, coord_pw AS pw, 'div' AS accesslevel FROM main_divisions WHERE coord_email is not null AND coord_pw is not null

UNION ALL

SELECT coord_email AS email, coord_pw AS pw, 'un' AS accesslevel FROM main_unions WHERE coord_email is not null AND coord_pw is not null

UNION

SELECT coord_email AS email, coord_pw AS pw, 'conf' AS accesslevel FROM main_conferences WHERE coord_email is not null AND coord_pw is not null

UNION

SELECT email, pw, 'people' AS accesslevel FROM main_people WHERE email is not null AND pw is not null

UNION

SELECT email, pw, accesslevel FROM main_ext_logins WHERE email is not null AND pw is not null

When mail provider changes IP (eg. causing mail queue just filling up)

(NOW THAT WE HAVE SWITCHED TO USING Google Apps WE NO LONGER ARE SENDING VIA sh.zmailcloud.com)

Steps to fix:

1. Get IP of nslookup sh.zmailcloud.com
2. If ip has changed, stop postfix /etc/init.d/postfix stop
3. Take new ip and update "relayhost" in /etc/postfix/main.cf
4. Edit /etc/postfix/sasl_passwd to add new ip
5. Delete old encoded sasl database: rm /etc/postfix/sasl_passwd.db
6. Generate new encoded database: postmap hash:/etc/postfix/sasl_passwd
7. Start postfix
8. Send test message by: mail -s testing john@sharehim.org. Next, type a line of text and then to send, CTRL+d
9. Verify log file looks correct
10. Verify email made it though
11. If good, then lets try to resend and flush the queue: postqueue -f
12. View log file to see if messages start going out OK while checking status of postqueue -p to verify queue is decreasing

Server

Installing updates

For Debian-based distributions do the following, all as root:

• apt-get update
• apt-get -dy dist-upgrade

After all packages have been downloaded successfully, then perform the actual install:

• apt-get dist-upgrade

Syncing between Linux and Google Drive/S3/other cloud services with rclone

• Download rclone from http://rclone.org/downloads/
• Extract and put "rclone" in /usr/local/bin/ (and give it execute permissions)
• Run "rclone config" and create a new remote.
  • Name: type an appropriate name
  • Client ID: leave blank
  • Client secret: leave blank
  • Use auto config?: no
  • Open the link it provides in a browser and paste the resulting code
• Test with eg. "rclone lsd <name>:" to list root content
  • If you get the error "The domain policy has disabled third-party Drive apps" following these directions: http://stackoverflow.com/a/14502443/2404541

It can even sync between two cloud services (= remotes). We use it to backup S3 to Google Drive, as well as backing up tn1 to Google Drive.

See also http://wiki.linuxquestions.org/wiki/Rsync_with_Google_Drive

Backup

Files backed up on webserver

• /etc
• /home
• /root
• /usr/local/bin
• /var/log
• /var/www

Using CrashPlan. See installation and managing instructions at https://support.code42.com/CrashPlan/4/Configuring/Using_CrashPlan_On_A_Headless_Computer (maybe also https://www.liquidstate.net/installing-crashplan-on-a-headless-linux-server/)

Files backed up on tn1.sharehim.org

• rclone sync /var/www/ "SharehimGDrive:/backup/tn1_server/-var-www/" -v
• rclone sync /storage/ "SharehimGDrive:/backup/tn1_server/-storage/" -v