Merge pull request #166 from ClaytonWWilson/master

Merge from master

Lambdas/Lists/Chain/src/README.md

@@ -0,0 +1,16 @@
+# Chain Module
+Deals with information concerning store chains supported by the product populating and product serving Lambdas.
+
+### ChainGET
+#### Expected request body:
+N/A
+
+#### Expected query parameters:
+  - id
+    - Used for specifying which chain to retrieve
+    - Valid values: -1,1<n<{num_chains}
+    
+#### Inputs and outputs:
+  - id = -1: Returns an array if chainIDs (Integers)
+  - id = 1<n<{num_chains}: Returns a [Chain object](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/Chain/src/Chain.java) for the chain with chainID=id
+

Lambdas/Lists/Item/README.md

@@ -0,0 +1,18 @@
+# Item Module
+Deals with retrieving product information populated by the scripts at https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/APIs and https://github.com/ClaytonWWilson/Listify/tree/master/Lambdas/Scraping.
+
+### ItemGET
+#### Expected request body:
+N/A
+
+#### Expected query parameters:
+  - id
+    - Used for specifying which product to retrieve. This is not the upc, but is our own internal id.
+    - Valid values: 1<n<{num_products}*
+    
+#### Inputs and outputs:
+  - id = 1<n<{num_products}*: Returns an [Item object](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/Item/src/Item.java) for the products with productID=id \
+
+\*Not all number in this range are valid because a seemingly random subset were removed by deduplication scripts and during new product addition testing.
+
+  

Lambdas/Lists/ItemSearch/README.md

@@ -0,0 +1,17 @@
+# ItemSearch Module
+Deals with retrieving aggregate product information meeting certain naming criterion from among that populated by the scripts at https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/APIs and https://github.com/ClaytonWWilson/Listify/tree/master/Lambdas/Scraping.
+
+### ItemGET
+#### Expected request body:
+N/A
+
+#### Expected query parameters:
+  - id
+    - Used for specifying a substing of the product name that returned items should match
+    - Valid values: {any_string}
+    
+#### Inputs and outputs:
+  - id = {any string}: Returns an [ItemSearch object](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/ItemSearch/src/ItemSearch.java), which is basically an array of productIDs, for products whose name fits the regex criterion `name = ".*" + id + ".*"`
+    - Results are limited to the first 100 entries from the database (by earliest add time)
+
+  

Lambdas/Lists/List/README.md

@@ -0,0 +1,85 @@
+# List Module
+Deals with managing the user-facing list creation, deletion, retrieval, and renaming. The supporting classes here also serve as resources for other modules.
+
+### Notable
+[ListPermission.java](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/List/src/ListPermissions.java) contains the lambda-side translation from permissionLevel to actual permissions. Keys updated on the client side must also be updated here.
+
+### ListGET
+Retrieves details for an existing list
+#### Expected request body:
+N/A
+
+#### Expected query parameters:
+  - id
+    - Used for specifying which list to retrieve
+    - Valid values: -1,1<n<{num_lists}*
+    
+#### Inputs and outputs:
+  - id = -1: Returns an array if listIDs (Integers) the requesting user has Read access to
+  - id = 1<n<{num_lists}*: Returns a [List object](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/List/src/List.java) for the list with listID=id. ListGetter is developer friendly and proactively populates all of the [ItemEntry](https://github.com/ClaytonWWilson/Listify/blob/readmes/Lambdas/Lists/List/src/ItemEntry.java) objects in the list with their data.
+
+*Not all number in this range are valid because a seemingly random subset may be removed by user activity and testing.
+
+
+### ListPOST
+Creates a new list
+#### Minimum expected request body:
+```
+{
+  "name": "listName"
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - name: Provides the display name for the new list
+  
+#### Returns:
+  - listID: An integer which is the listID of the newly created list
+  
+  
+### ListPUT
+Updates a List's name
+
+#### Minimum expected request body:
+```
+{
+  "listID": 2,
+  "name": "newListName"
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - listID: Provides the listID of the list which shall receive the new name
+  - name: Provides the new display name for the list
+  
+#### Returns:
+null
+  
+### ListDELETE
+Deletes a list and cleans out dependent tables like ListSharee
+#### Minimum expected request body:
+```
+{
+  "listID": 3
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - listID: Provides the listID of the list to delete
+  
+#### Returns:
+null
+  
+  

Lambdas/Lists/ListEntry/README.md

@@ -0,0 +1,51 @@
+# ListEntry Module
+Deals with managing entries in a list. Note that there is no GET here because that is handled in ListGET see [List readme](https://github.com/ClaytonWWilson/Listify/tree/master/Lambdas/Lists/List)
+
+### ListEntryPOST
+Add a specified quanitity of an item to a list.
+#### Minimum expected request body:
+```
+{
+  "productID": 2,
+  "listID": 4,
+  "quantity": -3,
+  "purchased": false
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - productID: The productID to add to the list
+  - listID: The listID of the list to be modified
+  - quantity: The number of an item to add (may be negative) to the list
+  - purchased: Currently unused field with future potential for allowing user to check off items they have purchased
+  
+#### Returns:
+null
+  
+  
+### ListEntryDELETE
+Removes the specified product from the list
+
+#### Minimum expected request body:
+```
+{
+  "listID": 6,
+  "productID": 5
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - listID: The listID of the list to be modified
+  - productID: The productID to remove from the list
+  
+#### Returns:
+null
+  

Lambdas/Lists/ListShare/README.md

@@ -0,0 +1,49 @@
+# ListShare Module
+Deals with permissions for accessing Lists and performing functions on them. This enabled social-media-like functionality.
+
+### Concerns
+More information is public here than may be desirable. In the future, it may be worth reducing the amount of information served by this Lambda about users other than the requestor.
+
+
+### ListShareGET
+Retrieves a information on who may access a list and how they interact with it.
+#### Expected request body:
+N/A
+
+#### Expected query parameters:
+  - id
+    - Used for specifying which list to retrieve share information for
+    - Valid values: -1<n<{num_lists}*
+    
+#### Inputs and outputs:
+  - id = 1<n<{num_lists}*: Returns a [ListShare object](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/ListShare/src/ListShare.java) for the list with listID=id.
+
+*Not all number in this range are valid because a seemingly random subset may be removed by user activity and testing.
+
+
+### ListSharePUT
+Modifies a user's permissions to interact with a list.
+#### Minimum expected request body:
+```
+{
+  "listID": 3
+  "shareWithEmail": "email@mailserver.com"
+  "permissionLevel": 210
+
+}
+```
+Other fields may be populated and are simply ignored.
+
+#### Expected query parameters:
+N/A
+
+#### Inputs:
+  - listID: The listID of the list for which to alter permissions
+  - shareWithEmail: The email (already register with Cognito) of the user whose permissions are being changed.
+  - permissionLevel: A numerical representation of a user's permissions. This should be generated and interpreted by [ListShare.java](https://github.com/ClaytonWWilson/Listify/blob/master/Listify/app/src/main/java/com/example/listify/data/ListShare.java) on the client-side or it's server-side equivalent [ListPermissions.java](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/List/src/ListPermissions.java)
+  
+#### Returns:
+null
+
+#### Notes for ListSharePUT
+Unsharing is accomplished by sharing with permissionLevel = 0.

Lambdas/Lists/README.md

@@ -0,0 +1,15 @@
+# Client-Facing Lambdas
+Each subfolder (except `target`) is a module.
+
+NOTE: `src` contains the Core module
+
+## Building
+- The pom.xml has dependency information for all the Modules combined and all these dependencies are currently built into the Core module.
+- Other build configurations are done in IntelliJ with each module having a build definition.
+  - Each module relies the (extracted) output of Core.jar being included in its build.
+  - Other modules may be included as applicable.
+  - Some modules specifically include their own dependencies, but this is not strictly necessary since everything is (inefficiently) bundled into Core.
+  
+## Inclusion in Git
+- Compiled output should never be included in git.
+- IntelliJ's build files may not be properly included (or may just be hard to import.) Copying the folder directly between machines works.

Lambdas/Lists/User/README.md

@@ -0,0 +1,26 @@
+# User Module
+Wraps certain Cognito functionality to provide greater ease of use for the developer than always calling Cognitor directly and using service hooks.
+
+### UserGET
+Retrieves information on a user's cognitoID or email. This Lambda exists for ease of use in retrieving/converting between returns.
+    
+#### Potential Inputs (choose 1)
+  - (query paramter) id: A user email or a Cognito id. Note: relies on the assumption that Cognito ids do not have "@" and emails do
+  - (body element) emailToCheck: A user email
+  - No input (or null/empty string inputs)
+  
+  
+#### Returns:
+  - A User object (if asking for an email, the Cognito id may be incorrect)
+    - No input results in only the Cognito id being populated with the requestor's id.
+    - Should something about the contract be broken, or the user not exist, a null, or null-populated, value will be returned
+
+
+### UserDELETE
+Deletes a user and their data. This Lambda exists to ease database cleanup and sending a farewell email.
+
+#### Inputs:
+None, this only uses the Congito id collected by AWS Gateway
+
+#### Returns:
+null

Lambdas/Lists/User/src/UserDeleter.java

@@ -40,6 +40,7 @@ public class UserDeleter implements CallHandler {
     private final String DELETE_LISTS = "DELETE FROM List WHERE (owner = ?);";
     private final String DELETE_LIST_SHARES = "DELETE FROM ListSharee WHERE (listID = ?);";
     private final String DELETE_LIST_ACCESS = "DELETE FROM ListSharee WHERE (userID = ?);";
+    private final String DELETE_PROFILE_PICTURE = "DELETE FROM Pictures WHERE (userID = ?);";
 
     public UserDeleter(Connection connection, String cognitoID) {
         this.connection = connection;
@@ -137,6 +138,11 @@ public class UserDeleter implements CallHandler {
         System.out.println(statement);
         statement.executeUpdate();
 
+        statement = connection.prepareStatement(DELETE_PROFILE_PICTURE);
+        statement.setString(1, cognitoID);
+        System.out.println(statement);
+        statement.executeUpdate();
+
         connection.commit();
 
         return null;

Lambdas/Lists/src/README.md

@@ -0,0 +1,36 @@
+# Core Module
+Wraps inner business functionality and abstracts certain environment interactions such as opening/closing the database connection and processing the API Gateway input json.
+
+#### Expected input json:
+The following is created by APIGateway along the lines of [this](https://github.com/ClaytonWWilson/Listify/blob/master/Tooling/body_and_auth_mapping.json) definition
+```
+{
+  "body": {jsonizedPOSTEDObject},
+  "params": {
+      "querystring": {
+          "queryParamifApplicable": "paramValue"
+      }
+   },
+   "context": {
+       "sub": "cognitoID"
+   }
+}
+```
+
+`querystring` will only have subcomponents if there are query parameters
+
+
+`{jsonizedPOSTEDObject}` will be something like:
+```
+{
+    "var1": 1,
+    "var2": "string"
+}
+```
+
+#### Module Contract:
+For the module contract see [CallHandler](https://github.com/ClaytonWWilson/Listify/blob/master/Lambdas/Lists/src/main/java/CallHandler.java)
+
+#### Returns
+Business logic return or error is appropriate
+

Lambdas/README.md

@@ -0,0 +1,35 @@
+
+## Components
+### APIs
+Contains logic to retrieve data from merchant APIs.
+
+### Lists
+Contains client-facing Lambdas which are typically a rather thin wrapper over the database (MariaDB on AWS's RDS service) and other AWS services.
+#### General Client-facing Serverside Flow
+```
+Client -> API Gateway endpoint -> Lambda  -> (same) API Gateway endpoint -> (same) Client
+            |         .           |    .
+           \ /       / \         \ /  / \
+            '         |           '    |  
+           AWS Cognito          MariaDB instance/Other AWS services(SES, Cognito)
+```
+
+### Scraping
+Contains logic to orchestrate runs using a scraping utility to collect merchant data and store it in the database
+
+
+
+
+## Common Troubleshooting
+#### Logs
+- Logs may be written to by printing to the stdout. (Fancier logging is also possible, but currently not implemented)
+- Logs are avalaible on AWS's Cloudwatch under Log Groups -> {Lambda name}
+- Logs typically persist after deletion of the associated Lambda
+
+#### Incomplete runs
+Often if a run fails to complete, it is because the standard allocated time and memory/cpu is inadaquate.
+- Symptoms:
+  - The running time (available in the Cloudwatch logs) is almost exactly the maximum time.
+- Resolutions:
+  - Time may be increased.
+  - CPU may be increased. CPU is proportional to memory, so CPU is increased by increasing memory allocation.

Tooling/README.md

@@ -0,0 +1,68 @@
+# Tooling
+
+This folder contains supporting scripts to automate the serverless backend resource creation/updating.
+
+
+## Scope
+The scripts here will create and update AWS Lambdas and their paired API Gateways as well as attaching appropriate IAM roles to the Lambdas and Cognito integration to the API Gateway.
+
+Covered:
+ - AWS Lambdas
+ - AWS API Gateway
+ - AWS IAM <-> AWS Lambda interaction
+ - AWS Cognito <-> API Gateway interaction
+ 
+Not covered:
+ - Verification of Lambda correctness: These scripts will in no way verify that the built jar is correct.
+ 
+
+
+## Files
+
+#### EndpointSetup.sh
+  - Purpose: Creates a new AWS Lambda, and, if needed, a new API Gateway endpoint and HTTP method on the endpoint. The names are autogenerated based on the method and the input name.
+  - Inputs:
+    - Resource name: Name of the resource (e.x. List).
+    - HTTP method: HTTP method to use to access this particular Lambda. Will be appended to resource name to get expected Java class (e.x. resourceGET)
+    - Resource jar: The script will search the project directory for a jar with the given resource name (resource.jar). The jar should contain all method classes for the resource as well as any dependencies.
+    
+#### LambdaUpdate.sh
+  - Purpose: Refreshes the jar attached to a specific Lambda.
+  - Inputs:
+    - Resource name: Name of the resource (e.x. List).
+    - HTTP method: HTTP method to use to access this particular Lambda. Will be appended to resource name to get expected Java class (e.x. resourceGET)
+    
+#### VarSetup.sh
+ - Purpose: Sets up some constants related to the AWS account and basic resources (e.x. broad API gateway id, Cognito pool). Collects constants for easy modification
+ 
+ 
+ 
+ 
+## Running
+ 
+```
+$ bash Tooling/EndpointSetup.sh 
+Creating a Gateway/Lambda pair.
+Please enter base function name: List
+Please enter method(GET, POST, etc.): PUT
+
+```
+
+```
+$ bash Tooling/LambdaUpdate.sh 
+Updating a Lambda.
+Please enter base function name: List
+Please enter method(GET, POST, etc.): PUT
+Update successful.
+```
+
+VarSetup.sh should not need to be called directly since the other scripts `source` it.
+ 
+## Debugging
+ 
+ VarSetup.sh has a `DEBUGFILE` constant in it. By default it is `/dev/null`, but you can change it to something else to get debugging output.
+ 
+## Notes
+- Many configurations are stored as constants in the scripts and can be modified. Most of these constants should be in VarSetup.sh, but some may be elsewhere.
+- Sometimes the allocated memory and/or runtime is insufficient and must be adjusted upwards.
+- If in Windows Subsystem for Linux, you may have to remove '\r' characters that Windows added to the .sh files.