<iframe src="https://www.googletagmanager.com/ns.html?id=GTM-M74D8PB" height="0" width="0" style="display:none;visibility:hidden">
Loading
Skip to NavigationSkip to Main Content
0D54z00008YhNIwCANOkta Identity EngineWorkflowsAnswered2023-04-27T15:25:18.000Z2022-12-22T17:17:19.000Z2023-04-27T15:25:18.000Z

NickC.22997 (Customer) asked a question.

Uploading an Exported CSV File to an S3 Bucket

Has anyone successfully utilized the "Custom API Action" AWS S3 Card to Upload a File to an S3 Bucket? I am exporting a CSV File from an OKTA WF Table, and I need to upload that generated CSV File to an S3 Bucket. I have the AWS S3 Connector working, and it can 'see' the Bucket, but I'm not able to find a way to Upload a File to it via 'PutObject'.

 

I was really hoping the "Copy Object to S3 Bucket" Card was going to do this, but this is just for Copying an object from 1 S3 Bucket to another S3 Bucket (UGH!)


NickC.22997 likes this.
  • TimL.58332 (Workflows)

    @NickC.22997 (Customer)​ 

     

    The S3 card doesn't have an "upload" option to upload a binary file (type: "File" in Workflows) into S3 which I suspect is why you are attempting to leverage Custom API Action (CAPIA). CAPIA doesn't have the ability to leverage type "File" so you are stuck.

     

    You may have success leveraging the "File" cards upload or multipart upload. You will need to define the Auth in the headers based on the AWS documentation example I see.

     

    https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

     

    Notice the example has an Authorization Header.

     

    I don't have any confirmation of this ask working but it appears to be feasible.

    Expand Post
    Selected as Best
  • TimL.58332 (Workflows)

    @NickC.22997 (Customer)​ 

     

    The S3 card doesn't have an "upload" option to upload a binary file (type: "File" in Workflows) into S3 which I suspect is why you are attempting to leverage Custom API Action (CAPIA). CAPIA doesn't have the ability to leverage type "File" so you are stuck.

     

    You may have success leveraging the "File" cards upload or multipart upload. You will need to define the Auth in the headers based on the AWS documentation example I see.

     

    https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

     

    Notice the example has an Authorization Header.

     

    I don't have any confirmation of this ask working but it appears to be feasible.

    Expand Post
    Selected as Best
  • NickC.22997 (Customer)

    I actually wanted to provide an update since I've been able to play around with this.

     

    I was able to successfully Upload (PUT) a CSV File to an S3 Bucket using the "Custom API Action" AWS S3 Card!

     

    This is sort of a workaround/nuanced way of accomplishing it, but it works. Here is a high-level of my Flow:

     

    • Get Group Members & Profile Data from 2 OKTA Groups
      • Records are streamed to a Helper Flow which populates an OKTA Table
    • Back in the main Flow, I use a Table "Search Rows" Card to generate a listing of all the Rows (LDAP, FirstName, LastName, Email, etc.)
      • NOTE: I had to use 2 Table "Search Rows" Cards since my Table is over 3,500 Rows. This Card only returns a max of 3,500 Rows, so the second card I put 3,500 as the 'Offset' input parameter
    • The complete Rows Results are sent to an Object Construct card
    • The Output of that is sent to a Compose card which now contains the complete Text of all the Rows.
    • I then do a series of Text manipulation (basically removing characters) using multiple, sequential Text "Replace" Cards
    • Once I get the Text how it needs to be (formatted like a CSV file), I finally call the AWS S3 "Custom API Action" Card
      • Options:
        • Request Type: PUT
      • Request:
        • Bucket: Name of the S3 Bucket
        • Relative URL: /MyTestFile.csv
        • Query: {}
        • Headers: { "Content-Type": "text/csv" }
      • Body: The output of the final, manipulated Text (formatted like a CSV file)
        • Example:

    "First_Name","Last_Name","Email"

    "John","Doe","JohnDoe@aol.com"

    "Jane","Doe","JaneDoe@juno.com"

    • DONE

     

    Obviously being able to leverage the File "Upload" Card would be A LOT easier and more efficient as you'd be able to just Export the Table as a CSV File and Upload the actual File, but the necessary Signing data for the Authorization parameters of an AWS REST API call to do the PutObject seems very complex (and I just didn't have the time to try it out, etc.). Instead, the AWS S3 Connector takes care of all that work 😀

     

    Here are a few snap-shops showing a successful test using the AWS S3 "Custom API Action" Card:

    Image 12-27-22 at 9.40 AM (1)Image 12-27-22 at 9.40 AMImage 12-27-22 at 9.46 AMScreenshot 2022-12-27 at 10.56.03 AMImage 12-27-22 at 9.42 AM

    Expand Post
  • SangL.04191 (Customer)

    I ran into this issue as well and finally found a resolution that utilizes the File upload card in Workflows (as Tim mentioned above)!

     

    You'll first need to do some configurations on the AWS-end to setup the roles, policies, buckets, API gateway configs. Follow these instructions (https://www.youtube.com/watch?v=PxO8L9s3xOw)

     

    After, set up your workflow to upload the files via Workflows. There's no need to utilize the S3 Connections in Workflows.

     

    image

    Expand Post
This question is closed.
Loading
Uploading an Exported CSV File to an S3 Bucket